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J. INTRODUCTION 


This thesis deals with detection and estimation using spatially separated sensors. 
A typical practical situation is a surveillance system [1] in which a large number of 
sensors monitor some region of space, earth or sea and report their findings to a global 
processor. The sensors themselves may use thermal, acoustic or infrared effects to form 
their observations. The global processor performs some processing on the data to come 
with a decision or for taking actions. Because of many considerations such as 
bandwidth communication limitations, time delay or because the amount of 
information is too massive to be processed by a single processor, the processing 1s 
carried out on many levels. As an example consider the case of distributed detection. 
Detection is performed at the sensor level and at the fusion center. 

Due to the loss of information in the local processing, the overall performance 
degrades. However a great communication bandwidth reduction results. If the 
communication channels can support more information flow, then it is wise to perform 
“softer” processing at the local level, to send more information to the fusion center, and 
to use the information available there effectively. 

The purpose of this chapter is to define the Distributed Signal Processing (DSP) 
problem in general and to show some reasons and situations in which it replaces 
Centralized Signal Processing (CSP) techniques. We then will review the status of the 
research on Decentralized Detection (DD) problem, one of the basic problems of DSP. 


Finally the contributions and organization of this thesis are described. 


A. OVERVIEW 

Classical (Centralized) Signal Processing (CSP) assumes complete availability of 
all information (signals) at one central processor for processing (decision making, 
computing, detection, estimation, etc...), While this situation is realistic in sore cases, 
manv real world systems are too large for the classical processing to be practically 
applied. Power systems, detection networks, large manufacturing systems and military 
Organizations are among those systems in which total centralized signal processing is 
hard to apply. Some of the reasons and considerations for the limations of CSP are 
ol: 


is 


l. In large systems, each processor has partial information of some credibility. 
While total information is distributed in the whole system, total centeralization 
of the information at one processor is impractical, inconvenient or expensive 
due to limitations in the system’s communication channels, memory or 
computation and information capabilities. 


2. In some cases, processing speed is a bottleneck. Increasing local processing of 
the data at each processor and sending processed data to the next level of 
processors will help relieve the problem. 


3. When reliability of the system is of major concern, distributed processing may 
better tolerate various kinds of equipment failures. Less complex centralized 
processing is more easily shifted to a new location. 


4. In cases when security is a major problem, increasing local processing will 
decrease the information handled between the processors, so limit any other 
SYSLGM1 S sacecss to tne process. 


5. As the cost of computation has decreased dramatically relative to the cost of 
communication, it is advantageous to trade off increased computation for 
reduced communication. So in Distributed Sensor Networks (DSN) involving 
geographically distributed sensors that collect data, 11 may be more economical 
to locally process the data and send condensed summaries to other processors. 


Distributed Signal Processing (DSP), in contrast to CSP, has several processors 
that cooperate together to best achieve a global task according to some criterion. A 
basic problem in DSP, which has attracted much attention recently, is the 
Decentralized Detection (DD) problem (hypothesis testing). The DD problem will be 


a major concern in this thesis. A summary of its status is given in the following section. 


B. MOTIVATION 

There has been an increased interest in the DD problem since Tenney and 
Sandell introduced it in 1981 [5]. They extended the classical Bayesian formulation of 
the detection problem to distributed environments. Because their work was the 
ploneering one in DD and because we will refer to it often in this thesis , let us 
consider it now in some detail together with the Centralized Detection (CD) problem. 
Also, because detection is dealt with throughout a large portion of this thesis, we will 
make some remarks about the phenomena to be detected and about detection criterion. 
The Phenomena 

Consider observing a phenomena H of M possible states in order to determine 
which of them is true. For M = 2, the state Hg is called the null hypothesis and H, 


the alternative hypothesis. Their probabilities of occurrence 


P(H) )=Py, P(H,)=P, (1.1) 
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are assumed to be known. 
The Sensor Observations 

The phenomena H is observed by N sensors Sy Sosy . The sensor 
observations are Yj,Y5.--4.¥xy + The sensor observations have known conditional 


distributions 
PUY Yo-s¥x/Ho), PUY Yau-5¥yy /H, ). (1.2) 
Detection Criterion 
The function of the detection process is to make a decision, U 9? about which 
state of the phenomena is true. The optimality criterion is a function 


J: U, xH > &, | (1.3) 


that assigns to the event of deciding u; when A; is true a real number C., 1,J= 0,1, 


called the detection cost, so 
ee =u; ,H =H; )= C;.. (1.4) 
The objective of the decision rule will be to minimize the expected decision cost 
min E{J(u,H)}. (1.5) 
An important ratio in our analysis is the constant given by 


0 (Si0Sno) (1.6) 
Py (Co -Cy) | 


Van Trees [6] showed that the average decision cost is given by, 


R= Coo Po F Co, Py Po (Coy “Cin ) Pp -Py (Coy Cay Pa O20 
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Where P, and P, are the probability of false alarm! and probability of detection? 


respectively. At this point we will make the assumptions that 

Cree Cue (1.8) 
and 

Cio > Coo» (1.9) 
These assumptions implies that it is more costly to err than to make a correct decision. 
Equation (1.7) can then be written in the form: 


Re Rx Po 7 Son Py ae Po (Cig “Coo 
Py (Co) Ci ) P (Coy Cy ) 


Pra] Pi Cox Cu) (1.10) 


Ignoring positive constants that will not affect our analysis, the average decision cost 


R is given by 


R=1+CP,-P,. (1.11) 


1. The Centralized Detection (CD) Problem 
The problem of centralized binary hypothesis testing can be posed in its most 
general form as follows. For the structure of Figure 1.1 it 1s assumed that all sensor 
observations can be sent to one (central) location for processing. The function of the 


processor is to map the vector Y= [y¥) y,.. Vx |' into the decision space U&subo(0,1) 
Uy: X > (0,1) (1.12) 


as follows; 
0, Hy is declared to have been detected 
U, = (1.03) 


lL Ea wis declaredite have Deen detected. 


1 


lProbability of deciding U > ~0 while H, is true 
Probability of deciding U > 71 while H, is true 
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Figure 1.1 Centralized Detection. 
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Solution of the CD Problem 
The solution to the CD problem is [6] 
a) deterministic, so that the decision rule is a function of the observations 
YG Sy x ee On (1.14) 


b) a likelihood ratio test, 


Q, if A (Vp Yo-5¥x ) = C 


UYp¥y Yn )= LL LACS gmn¥u) <1 (1.15) 
where 
AAV Gorn) = SNe (1.16) 
LY Vas¥n (Hg) 
c) and the threshold t 1s given by 
t= Cs (1 


2. The Decentralized Detection (DD) Problem with Fusion 
Consider the structure of Figure 1.2 with H and Y being as before; the 
decisions UL, ,L,,... and Uy are sent to a fusion center. The activity of the fusion 


center is to make the global decision U, according to some preset fusion rule. 
U,: U, xU,..xU,; — (0,1). (1.18) 


In the DD problem with fusion it is required to design local decision rules U,,U,,... 
and U,; and a global fusion rule (1.18) so as to minimize the expected cost E(J(U, ,H)} 
incurred by deciding U, =i when H, is true. 

Choosing an AND fusion rule apriori, Tenney and Sandell solved this problem 
for N=2. They set the decision rule as U, =U, U, and optimized the local decision 


rules. 
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Figure 1.2 Decentralized Detection with Fusion. 
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Solution of DD Problem with Fusion 
The solution to the DD problem with fusion is 


ir deterministic 
eee tee Os) (1.19) 
and 
ee Oy (1.20) 


2. a likelihood ratio test for each detector 


— {° if A. (Yiy> G (1 21) 
: i if A, (Yiy< G 


where 
ve 
= St (1.22) 
ss f(g ale) 
3. with coupled thresholds t, and t, given by 
Eeitele say 
teen eae (1.23) 
Pr(D,, iY] ) 
and 
Preis 
t, = aes (1.24) 
Pr(D, lY> ) 


where Pr(F. ly; ) and Pr(D. IY; ) are respectively the conditional probability of 
false alarm and the conditional probability of detection of the 1, detector given 


the j,,, detector’s observation. 
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Equations (1.23 ) and (1.24 ) are two coupled functional equations in t, and t, . For 
general distributions, a functional expression for each of them in terms of its own 
observation and the other detector’s decision is impossible. We shall consider the 
complexity of these decision rules later. A special case of the DD problem is the case 


of conditionally independent sensor observations, 1.e. 


fly, /y, .H)= fly, /A) (1.25) 
and 
fly, /y, »H)= fy, /H). (1.26) 


In this case, the conditional probabilities in (1.23 ) and (1.24) reduce to 








P 
t, =C—# (1.27) 
Pao 
and 
P 
tf =c—t (1.28) 
Pal 


Equations (1.27) and (1.28) are two coupled algebraic equations in the form of 
ie 8, (ty) and t, =g, (t, ) (1.29) 


since P, and P,. depend on t,. This coupling represents cooperation between the two 
1 di 1 
sensors. The threshold equations are necessary conditions for optimality. There may 
be several local minima; each must be checked to assure the global minima. The 
threshold equations are strongly coupled for general cost assumptions. 
Tenney and Sandell came to the following conclusions: 


1. Increasing the signal-to-noise ratio improves the performance of the system. 
However a centralized system makes more efficient use of the increased 
information. 


a 


2. As the imbalance between the two detectors increases the performance 
improves. If the signal-to-noise ratio of one of the detectors goes to zero then 
the system decision is that of the other detector. This is equivalent to the 
performance of a CD system of the same signal-to-noise ratio. 


The case of conditionally independent observations has been considered by 
many authors. Sarma and Rao [7] extended Tenney and Sandell’s results to the case of 
three sensors. They assumed a majority logic fusion rule and evaluated the threshold 
settings for some specific cases. Chair and Varshney [8] considered the problem of 
optimal fusion of N local decisions from prespecified local decision rules. Their 
optimum fusion structure is a weighted sum of local decisions according to their 
reliabilities. Reibman and Nolte [9] optimize both local decision rules and the fusion 
rule under the assumption of identical local decision rules. The global decision is then k 
out of N. They optimize the local decision rule for each k ,k=1,2,...,N, then pick the 
value of k corresponding to the minimum decision cost. 

A sub-class of the DD problem with fusion, that will be referrd to as the 
“Second Opinion” problem, is the fusion of one’s observation with another’s decisions. 
An example of this is the second opinion in a medical examination, or even asking for 
legal advise. Ekchian {10} and Ekchian and Tenney [11] consider some specific 
topologies of this problem. Each decision maker has to make his decision based on his 
own observation and a predecessor's decisions. All the decision rules are likelihood 
ratio tests using the actual data. The thresholds are determined by incoming 
communication messages. The number of thresholds at each decision maker grows 
exponentially with the number of message inputs. Their results suggest putting the 
noisv sensor “up stream” in the detection network. 

Papastavrou and Athans [12] also consider the second opinion problem. They 
examine the structure of a primary decision maker, PDM, and a secondary decision 
maker, SDM ( a consultant ). The POM makes his decision based on his own 
observation if it 1s of good quality. If his observation is noisy, the PDM asks, at a 
communication cost, the opinion of the SDM. Being activated by the request of the 
PDM, the SDM sends his decision to the PDM or ignores the request if his 
observation is noisy. In either case the PDM has to make a final decision. Again the 
thresholds are coupled. The threshold of the PDM is determined bv the message of the 
SDM. 

This thesis 1s motivated mainly by three of the above works namely; 


1. Bayesian formulation of the DD problem by Tenney and Sandell [5]. 
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2. Extension of the DD problem to the Distributed Detection Networks by 
Ekchian. [10] | 

3. Extention of the DD problem to the case of correlated sensor observations by 

Lauer and Sandell [4]. 


o 


C. THE COMPLEXITY OF THE DD PROBLEM 

We saw that the DD problem can be solved optimally for conditionally 
independent sensor observations. If this condition does not hold local decisions are not 
likelihood ratio tests with constant thresholds. Tenney and Sandell show that for 
conditionally dependent observations, local decision rules are likelihood ratio tests but 
with data dependent thresholds (see e.g. (1.23 ) and (1.24 )). These two equations are 
coupled. This means that the observation of one sensor is necessary for the other 
sensors decision, which contradicts the principle of decentralization. In terms of the 
terminology of the Theory of Combinatorial Complexity [13], Tsitisiklis and Athans 
[14] show that 

1. The DD problem with independent observations is a polynomial time problem. 


2. The DD problem with dependent sensor observations in its simplest form is a 
nondeterministic polynomial NP-complete. This means that exhaustive 
enumeration is necessary to find the optimum local decision rules. Optimality 
may be an illusive goal. So, suboptimal solutions must be sought. 


A suboptimal solution to the problem for the case of AND fusion was considered 
by Lauer and Sandell [4]. Thev considered the case of known signals in correlated 
noise. They took as a suboptimal solution local decision rules which are likelihood 
tests but having constant, not data dependent, thresholds satisfying the necessary 


condition of optimality. These thresholds are given by the implicit equations: 


Pr(F, /T, ) 


Pr(D, /T, ) aa 


ar, j= C 


and 


Pr(F, /T, ) 


A,(T,)=C | 
a Ure eed), Aly ) 
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D. 


CONTRIBUTIONS OF THIS THESIS 
We have reviewed the complexity of the DD problem and its current status. The 


research reported here has significantly advanced this status in several important ways. 


Specifically the contributions of this thesis have been to : 


I. 


oe 


CIDA 


E, 


Answer the question of the optimum fusion rule at the fusion center for the 
case of two sensors. 


Specify the exact relation between the performance of the optimum fusion rule 
and the correlation coefficient between sensor observations. 


Solve the the second opinion decision problem. 


Solve the multi-level DD problem with fusion; i.e. detection with quantized 
sensor data for the known signal in noise case. 


Introduce the minimum risk quantizer. 
Grade the road between DD detection and CD detection. 
Optimallv design quantizers for minimum mean square estimation. 


Present an efficient procedure to calculate parameters of a large variety of 
quantizers. 


ORGANIZATION OF THE THESIS 
The thesis is organized as follows. In Chapter II we consider the problem of 


fusion in DD. Optimum detection with quantized sensor data is considered in Chapter 


III, where the Quantized Detection algorithm, QD, is introduced. Numerical examples 


to illustrate the algorithm are given in Chapter IV. Generalization to the case of vector 


observations is presented in Chapter V. Optimum regeneration of sensor observations 


from their quantized versions and another sensor observation is considered in Chapter 


VI. A summary of the thesis, conclusions and suggestions for future research are given 


in Chapter VII. Proofs to some equations and FORTRAN programs to calculate 


parameters of the minimum risk and the minimum distortion quantizers are given in 


the appendices. 
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I. OPTIMUM FUSION OF LOCAL DECISIONS 


In this chapter the important question of the optimum fusion rule will be 
answered. The relationship of the optimum fusion policy to the ratio of costs and the 


correlation coefficient between observations is determined. 


A. INTRODUCTION 

Distributed Detection with fusion 1s a two level optimization problem. The 
problem can be formulated in the following three wavs: 

1. Local Decision Optimization 

The first way is to select the fusion rule aprior: and optimize the local decision 

rules accordingly. Setting the activity of the fusion center as AND fusion, Tennev and 
Sandell [5] derived optimum local decision rules for a pair of spatially separated 
detectors with conditionally independent observations. They prove that local decision 
rules are simple likelihood ratio tests with constant thresholds. The thresholds are the 
solution of a pair of coupled algebraic equations that correspond to the global 
minimum of the detection cost function. They also show that for the case of correlated 
observations local decision rules are likelihood ratio tests but with data dependent 
thresholds. Functional solution of the threshold equations in the later case violates the 
principle of decentralization. Realizing the difficulty of the problem in the case of 
correlated observations, Lauer and Sandell [4] designed suboptimal local decisions for 
AND fusion. Their local decision rules are likelihood ratio tests with constant 
thresholds satisfving the necessary conditions of optimality. Kovatana [15] considered 
AND fusion for two detectors. Fefjar [16] compared AND to OR _ fusion for two 
detectors. He claimed that OR is better than AND. Stearns [17] contradicts Fefjar’s 
results. He showed by an example that OR combining is better for higher cost of 
missing the target while AND combining is better for higher cost of false alarms. 

2. Fusion Rule Optimization 

In the second formulation of the problem, local decision rules are set apriori. 

Optimization is carried out with respect to the fusion rules. An example of this 
situation could be factory built sensors that cannot be adjusted. Assuming local 
threshold settings Chair and Varshney [8] prove that for the case of conditionally 
independent sensor observations, the optimum fusion rule is a likelihood ratio test that 


sums local decisions weighted according to their reliability. 


2/7 


3. Global optimization of the Local decisions and the Fusion Rule 

The third formulation involves optimization at both levels. Here local 
decisions are optimized for every possible fusion rule. The optimum fusion rule is the 
One that minimizes cost. 

The main issue of this chapter is the global optimization of the DD system for 
general correlated observations. First we will state the main results for the case of N 
conditionally independent and identically distributed sensor observations. Then, the 
problem of fusing two local decisions of sensors with correlated observations 1s 


considered. 


B. GLOBAL OPTIMIZATION OF DISTRIBUTED DETECTION 
In CD all sensor observations are available at one central processor for detection. 
The decision rule in CD is a likelihood ratio test in the observations V,,Y5,..4,Yy. It 


declares H, is true if the likelihood ratio 
ISN o, 5) eee G (2.1) 


otherwise it will declare Hj to be true. 

In DD onlv local decisions are sent to the central processor ( fusion center). The 
objective of the fusion center is to mix ( fuse ) the local decisions into a single global 
decision with minimum decision cost. So given the local decisions the observation 
space of the fusion center consists of 2 discrete points. The activity of the fusion 
center is to divide this space into two decision regions Z, and Z,. The decision rule of 


the fusion center is a likelihood ratio test [S.] The fusion center declares H, is true if 
A (Uy ,Us,.5Uy, ) 2 C. (23) 


otherwise it will declare that Hg is true. In the special case of conditionally 
independent and identically distributed observations, the fusion rule is a k out of N 
rule. Reibman and Nolte [9] considered this problem. Assuming the same decision rule 
for every detector they optimize local decisions for every k, k= 1,2,...,N then pick the k 
with the minimum decision cost. 

If sensor observations are not conditionally independent, there is no guarantee 


that local decisions are simple likelihood ratio tests. The problem turns out to be NP- 


complete which needs exhaustive enumerations to find the optimum decision rules [14.] 
Moreover if sensor observations are not identically distributed, there are as many as 
228 possible fusion rules for the N sensor decisions. Any algorithm that goes through 
the entire fusion list optimizing local decisions will be impractical? for N 2 6. 

Our approach to avoid this exhaustive enumeration is the following: 


1. We assume that local decisions are likelihood ratio tests with constant 
thresholds. Again we emphasize that this assumption is valid only for 
conditionally independent observations, there is no guarantee that it is correct 
for correlated observations [5]. So the constant threshold likelihood ratio test is 
optimum for conditionally independent observations and perhaps suboptimum 
for correlated observations. However the solution tends to the optimum 
solution as the correlation coefficient tends to zero [4]. 


2. Those fusion rules which agree with the CD solution will be tested. The rest of 
the fusion rules will be disregarded. The meaning of this will be made clear in 
the following example. 


Let us consider the case of two sensors (N =2 ) in detail. To be explicit, 


consider detection of known signals in gaussian noise. The sensor observations are 


given by: 

Hy: yea t, i= 1,2 (2.3) 
and 

oe 3. = a. en) i= 1,2. (2.4) 


The a.’s are positive constants and WN [n, n,}' is vector of zero mean with 


covariance 


l 
K = ; (2-5) 
p 1 


where f is given by 


p= E{n,n,}. (2.6) 





A. computer that spends | ft second in every optimization process, will spend 
40000 years to determine the optimum fusion rule, for N = 6. 
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The threshold equation of the CD problem is given by [6], 
(a, -P a, )y, (a, - pa, )Y> =(a," ine -2p a, a5)/2+(1-p? Jlog(C) (2.7) 


which is a straight line in the y, y, plane. Figure 2.1 shows decision rules based only 
on D,, only on D,, both decision rules together, and the decision rule of CD. 

The global optimization requires optimizing local decision rules for every fusion 
rule then picking the fusion rule with minimum average cost. The observation space of 
the fusion center consists of four discrete points (0,0), (0,1), (1,0), (1,1). Any fusion 
rule divides this space into two decision regions Z, and Zp. There are 2* =16 methods 
to subdivide four points into two groups. Table 1 contains a list of those fusion rules. 
Some special cases for the detection problem are as follows. 


l. If Cig 7 ©, ie. the cost of missing the target is extremely high, the CD 
solution assigns all the observation space to Zp). The fusion center can perform 
the same. This is fusion rule one. 


2. Similarly if Cy, 7 &, the fusion center will always decide H,, this is fusion rule 
two. 


3. Ifa, = pa,, the CD will decide based only on y,. So will the fusion center. 
This is fusion rule three. This can only happen when a, 2 ay. 


4. Ifa, = pa,, the CD will decide based on y,. This ts fusion rule four. This can 
only happen when a, 2 a). 


The first two situations represent extreme conditions of C. The next two conditions 
deal with specific values of p. We also distinguish the following two cases. 
Case a 
-1 Sp S min( a, ,a, )/max( a, ,a, ). 
In this case the y, and y, intersections of the threshold equation (2.7 ) are of the same 
sign. 
Case b 
min( a, ,a, )/max( a, ,a,) <p 3 1. 

In this case the y, and y, intersections of the threshold equation are of different signs. 
We shall consider these intervals of p when we study the effect of correlation between 
sensor observations. 

The CD threshold in the y, y, plane suggests assigning the decision point (0,0) to 
Z, and (1,1) to Z,. The fusion rules from 5 to 14 do not do this. They either assign 


(0,0) to Z, or assign (1,1) to Z, or assign (0,0) and (1,1) to the same decision region. 
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Figure 2.1 Decision Rules. 
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TABLE 1 
EXHAUSTIVE FUSION LIST OF TWO DECISIONS 


2 Comment 


(0,0),(0,1),(1,0),(1,1) Co; 


. (0.1) | (0.0).(1,0).1.1) 


| (1,0) | (0.0).(0.1).C1.1) 


(1.1) (0.9).(1.0).(0,1) 





We will not consider these ten fusion rules further. The remaining two decision rules 
. are the AND fusion and the OR fusion. Let us now consider optimizing each of 
them. 
l. ’ AND’ Fusion © 
In AND fusion u, is given by: 


U, = U, U. (2.8) 


The individual rules are given by assigning y, to Z, if 
fee 1, i= 1,2. (2.9) 
Otherwise they assign it to Zp. 


The probability of detection P,(AND) and probability of false alarm PAAND) 


of the fusion center are given by: 


cS © 
Py(AND)= J. Jr Myta/H, ) dy, oy, (2.10) 
and 
CO © 
P-(AND)=f_ Jf _. fly,,y, /Hg ) dy, dy, (221) 
T; T, 


It has been shown in Chapter I that, to within positive multiplicative and 


additive constants, the average decision cost is given by 
Pe | + CP.-P.,. (2512) 


Substituting for P, and P, in (2.12) from (2.10) and (2.11) expresses R(AND) as a 


function of T, and T,. The necessary conditions for optimality are 
eexo 1, = Oandd R/OT, =0, (2213) 


which can be written in the forms: 
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CO oO 
C Jp NTpy,/Hg) dy, = J NTy,¥2/H, ) dy, (2.14) 
qT, a 


and 
cO CO 
C Jy, fy,,T, /Hg) dy, = 7 fty,,T, /H, ) dy,. (2.15) 
Applying Bayes rule and rearranging terms, one can write (2.14) and (2.15) as follows: 


o @) 
J T K(y,/T, #Hg) dy, 
A, (T, ) = C —————_—_ (2.16) 


J T f(y,/T, ,H)) dy, 
2 


and 


oe 
je (yy /T, Fo) dy, 
A, (T,)= C —_______ (2.17) 


To insure minima the Hessian matrix of R with respect to T, and T, 


(2.18) 


(2.19) 


. Rid T; © 7 Roald ie | 
H 


~{ 42 oe 2 
@R/OT,9T, R/T, 


must be positive definite. Optimum threshold settings T, and T, are the solution of 
(2.16) and (2.17) that corresponds to the global minima, so all possible solutions of 
(2.16) and (2.17) must be tried. The coupling between (2.16) and (2.17) to determine 
the thresholds represents the cooperation that can occur between the two local 


detectors to mininiize the overall decision cost. 
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2. “OR’ Fusion 
The decision of the OR fusion is given by 


uy =u, + Uy-U, UL. (2.20) 


The probability of detection P, (OR) and probability of false alarm P, (OR) are given 
by 


P. HOR)= L- pt p2 f(y .¥> /H, ) dy, dy, (2.21) 
and 
a (2 , 
PA(OR)=1- J Je fypy2 /Hg ) dy, dy, (2.22) 


while the necessary conditions for optimality are 


qT; af 
eee (2.23) 
2 
J, f{y,/T, ,H,) dy, 


and 


i 
l 
Jo, Myy/T2 Ho) dy, 
a (2.24) 
l 
Ves f(y, /T, ,H) dy, 


Again the Hessian matrix must be positive definite. 
3. Solution of the Nonlinear Threshold Equations 
The pair of coupled equations (2.16), (2.17) for the AND fusion and (2.23) 
and (2.24) for the OR fusion can be solved using Max’s technique [18]. The technique 
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is summarized as follow: pick a value of T, and calculate T, from (2.16) or (2.23 ). If 
the calculated value of T, does not agree with that value calculated from (2.17) or 
(2.24) then T, must be chosen again. This approach is time consuming. Another 
approach is the method of successive substitution [19]. We first put the two equations 
in the form 


T1147 GT) » 7 T 


= F(T 2.25 
Te4+} K 2k+] They ee 


2k 


then start with a reasonable guess for (T, )g and (T, )g. A suitable initial guess is the 
locally optimum solutions, i.e. the thresholds that would optimize the detection if each 


sensor works alone. These will be denoted by T,,, and T,,,. For known signals in 


0 
gaussian noise these are 


(T. jg =a. /2 +log(C)/a. . (2.26) 


4. Numerical Results 
We have solved the threshold equations for both fusion rules fora, =1.7 and 
a, = 2.3 for several values of p and C. 
To compare AND and OR fusion, define K as the ratio of the AND cost to 
the OR cost. 


1+ CP.(AND)- P, (AND 
K = See ee (2.27) 
| + C P,(OR) - P, (OR) 


We have also computed the Receiver Operating Characteristic? (ROC) curves of 
classical conymunication theory [20] for each fusion rule. | 

Figure 2.2 shows the rauo K as a function of C for p = 0, 0.2 , 0.4. The 
figure shows that AND fusion is optimum for C 2 1! and OR fusion is optimum for 
lower values of C. The same is clear from Figure 2.3; ROC curves of AND fusion are 
above those of OR fusion for C 2 1 and lower otherwise. The performance difference 
becomes smaller as the correlation coefficient increases. Also the figures show that the 
performance degrades for both fusion rules as p tends to one. This is in sharp contrast 


to CD which has perfect detection for p= 1. 


ae as a function of P, 
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The same effects can be concluded from Figure 2.4 and Figure 2.5. Figure 2.4 
shows the ratio K as a function of C for a, =1 and a, =2 and for p = 0, 0.25, 0.5. 
Figure 2.5 shows the ROC curves for both fusion rules for the same case. The figure 
shows that AND fusion is optimum for C 2 1 and OR fusion is optimum for lower 


values of C, 


C. THE EFFECT OF CORRELATION BETWEEN SENSOR OBSERVATIONS 
So far we have answered the question of the optimum fusion rule. For C 2 1 
AND fusion is optimum. Let us now examine the effect of the correlation coefficient p 
on the performance of AND fusion for C 2 I (its range of superiority). We assume 
without loss of generality that a, is greater than a,. The two necessary conditions for 
optimality of AND fusion are (2.16) and (2.17). For the problem of known signal in 


gaussian noise these can be written as: 


ion i 
erfe J —2—— 
J (1-p*) 


a= CC —— 2.28 
ou T, -a, -p (7, -a, ) silt 
erfc <a ea 
J (1-p*) 
and 
erfc mStar \ 
oa ce 2 fe (2.29) 
erfe -ay -p (T, a, fe 
- . p’) 


Notice that C appears only as a multiplicative constant in the two equations. The role 
of p is not that obvious. Examining the two equations leads to the following insights 


about the role of p: 


lL. T, =-© and T, =T,,, is a solution. This corresponds to the decision rule of 
D,. 

2. T, =-© and T, =T,,, is a solution. This corresponds to the decision rule of 
DIE 
l 


3. Ifa, is greater than a, we expect the performance of D, alone to be better than 
that of D, alone and that of the selfish decision rule in which each detector tries 
to minimize its own detection cost, not the system decision cost, by using T,,,, 


T 10° 
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Figure 2.5 ROC for AND and OR Fusion Rules 
a, = 1a. = Om 
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We now prove three lemmas concerning these equations. 
Il. Lemma 1. 
< 
Forp s ay /a., 
< 
q, as Tito 
and 


T, = Too 


where T,,, is the optimum threshold of the ith detector operating alone. 
Proof: 
Since the argument of the complement of the error function in each 


denominator 1s less than the argument in the corresponding numerator, the 
fraction is always less than one. This implies that 


eG le 
2. Lemma 2 
For p =a, /a,, the only possible solution of (2.28) and (2.29) ts: 


Sa 


2 2lo 


and 
qT, =.0, 
Proof: 


For p =a, /a, equation (2.29) becomes 
ee = ee (2.30) 


The corresponding value of T, is T, =-®. 
3. Lemma 3 
For p 2 a, /a, the optimum solution for T, and T, is: 


= .0 
1 


and 
T, = Tyo: 
This means that the decision of the optimum AND fusion is that of D,. 
Proof: 


Recall that the CD threshold line divides the observation space into two 
decision regions. For positive signals the following inequality is satisfied in the 
region to the right of the Ciouine: 


CACY aoa peat eee : (2.31) 
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The converse of this inequality is true in the left region. The decision region Z, 
of any other decision rule contains areas from the right and from the left of the 
CD line. Areas to the right will have a negative contribution to the decision 
cost while areas to the left will have positive contributions. Now assume that T 

and T, , where both are finite, satisfy the necessary condition (2.28) and (2.29). 
We shall prove that they cannot correspond to the optimum solution. The 
finite point ( T, ,T. ) either lies to the left or to the nght of the CD threshold 
line as shown in Figure 2.6 a and b respectively. In Figure 2.6 a the intersection 
of the CD line with the line y, = T, is a better solution since it excludes an 
area in which C fly, ,y, /Hg ) 1s greater than f(y, , y, /H, ). A better solution 
than this has the same T, but with T, = -© since the added area has negative 
contribution to the cost. In Figure 2.6 b, T, =-® and T’, is a better solution 
than T, and T., since the added area has a negative contribution to the cost. 
In both cases T, =-© is the optimum solution and the corresponding optimum 
value of T, is T,,.- 


As a result of the above three lemmas it is clear that 


1. Any solution of the necessary conditions must satisfy 
qT, : io and 
qT, sd Tt 


2. The performance of the AND fusion saturates to that of D, alone for p 2 
a,/a,. We might recall that the threshold line of the CD system changes slope 
at that value of p. We will refer to this value of p by p_. This result is in 
contradiction with Lauer and Sandell’s results [4] which shows performance 
continuing to degrade with increasing p for 


p 2 Dor: 
Limiting behavior for = p-l1. 
For p =-1 the joint probability density function f(y, sy, /Hg ) has values only on the 
line y, =-y,- So any threshold values T, and T, such that T, =-T, will produce 
AND fusion with zero probability of false alarm. This can be visualized from Figure 


2.7 . Consequently, P g Will be given by 


Pa — 0.5 eric 1, “a. bos erfc ! ae, } ; (2-37) 


Maximizing P, with respect to T, yields 
1, =(a, -a,)/2. (2233) 


For the special case of equal SNR sensors, a: 
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Figure 2.6 Optimum T, and T, for p = a, /ay. 
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Figure 2.7 Centralized Threshold Line For Different 
Values of p. 
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D. NUMERICAL RESULTS 

The average decision costs vs p for a, =a,=2, and C= 1 are shown in Figure 2.8. 
Threshold values T, and T, vs p for the same case are shown in Figure 2.9. Figures 
2.10 and 2.11 show the same for C= 10. 

These four figures for the case of equal signal-to-noise ratio show that the two 
detectors cooperate with each other using the same decision rule ( equal thresholds ). 
Their threshold is an increasing function of p. The limit of this threshold as p — -1 1s 
zero. This behavior agrees with (2.33). The limit of the threshold as p — 1 is T,,. This 
is because for p — | the two systems have identical observations. 

The detection cost curves show that the cost is an increasing function of p. The 
curve of the AND fusion has the same shape as the curve of the CD system. Both 
systems attain their best performance at p=-1l. They have the same worst performance 
for p= 1. 

Figures 2.12 and 2.13 represent the case of unequal SNR sensors for C=1. 


Figures 2.14 and 2.15 show the same for C= 10. 
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Figure 2.8 Average Decision Costs for Equal SNRs,C= 1. 


47 


1.0 ° 


0.5 


Ors 


== ee) 





Figure 2.9 Threshold Value for Equalisiv ns Gol 


48 


1.0 


COST 


LO 


DBZ 
2 
S 
= 
© 
AND D 
N 
S 
Ot | ! | 
mete, mols 0 Gr 
p 


Figure 2.10 


Average Decision Costs for Equal SNRs,C= 10. 
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Figure 2.11 Threshold Value for Equal SNRs,C = 10. 
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Figure 2.12 Average Decision Costs for Unequal SNRs,C= 1. 
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Figure 2.14 Average Decision Costs for Unequal SNRs,C = 10. 
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Figure 2.15 Threshold Value for Unequal SNRs,C= 10. 
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These four figures for the case of unbalanced sensors show that, the two 
detectors cooperate using different thresholds. The threshold of the higher signal-to- 
noise detector is an increasing function of p while the other threshold is a decreasing 
function of p. 

The cost curves show that the fusion rule has its best performance at p =-l. 
Both DD and CD have their worst performance at p= p.. For p 2 p. the 
performance of the optimum fusion rule is the same as the detector of the higher 
signal-to-noise ratio. Recall that CD system has perfect detection for p = 1 when the 
SNR’s are unequal. As C increases the average cost of each system increases. This can 
be explained from the expression for R in which the probability of false alarm is 
weighted by C. 


E. DISCUSSION AND CONCLUSIONS 

We have shown that the optimum fusion rule is determined by the ratio of costs 
and the apriori probabilities. For equal error costs AND and OR fusion rules are 
equivalent. This is not surprising since each system turns out to be the minimum 
probability of error detector; thresholds are adjusted such that 1 -P,= P, It might also 
be noted: that the optimality of the fusion rule is independent of the correlation 
coefficient and the signal-to-noise ratio in this case. We also note that the detection 
cost of the optimum fusion rule has its minimum value at p=-1I. [t has its maximum 
value at p=a,/a,. The performance saturates at the cost of decision of the detector of 
higher SNR. In the interval (p é€ [a,/a,,1]), the optimum fusion rule ignores the 
decision of the detector of lower SNR. As a good dynamical example that agrees with 
this result is the switched diversity combiner [21] in fading environments and its 
centralized counterpart, the maximum ratio diversity combiner [22]. Recall that for 
unequal SNRs the performance of the CD system improves in this interval and has 
perfect detection for p =1. Also it is important to note that the optimum thresholds 
of the individual observers are not the same as if they were operating independently, 
but must be determined by simultaneous solution of two coupled nonlinear equations. 
This represents the cooperation between the two detectors to work as a team. Lastly 
the performance difference between CD and DD is due to the information loss in local 
data processing. However DD has fewer requirements on the communication channel 
in contrast to CD which requires infinite bandwidth. A compromise between these two 


extremes is to allow more information than just decisions to be sent to the fusion 
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center. This is the concept behind the Quantized Detection algorithm cc 
following two chapters. 
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II. DETECTION USING QUANTIZED SENSOR OBSERVATIONS 


A. INTRODUCTION 

So far detection with sensor observations has been described using two methods. 
In the first method all sensor observations are sent to some central processor which 
makes a decision based on a likelihood ratio test. In the second method only local 
decisions are sent to the central processor which fuses these decisions into a global 
decision. While the first method is very easy to design it requires in principle infinite 
bandwidth communication channels. The second method requires onlv one information 
bit per detection. Detection with quantized sensor observations will be introduced in 
this chapter. The main goal of the chapter 1s to grade the road from the DD problem 
to the CD problem. It will be referred to by Quantized Detection, QD. The 
performance improvement of the DD problem will be traced as the amount of 
information delivered to the fusion center increases. 

First let us consider the problem of the Primary Decision Maker (PDM) and its 
quantized second opinion (consultant). We will prove three theorems concerning the 
decision rule of the PDM. Then fusion of two quantized observations of an arbitrary 
number of levels will be considered. As a special case, fusion of two sensor 
observations, one quantized to N levels and the other to N+1 levels, will be proven 
equivalent to the PDM and an N-level quantizer. Comparison between different 


configurations will follow. 


B. TEAM DECISION OF A PRIMARY DECISION MAKER AND A SECOND 
OPINION QUANTIZER. 


1. Formulation of the PDM Problem 
Consider the structure of Figure 3.1 in which yy is quantized into y, F by the 


quantization rule a@ of N levels. 
a: Y, Be OY pas (3.1) 


The primary decision maker will make his decision u >» about the phenomena H based 


on its own observation y, and the quantized observation Vig 


oT 


PHENOMENA 


ee 


y 1 \ y2 


v 


1 
Q1 as PDM uo 


Figure 3.1 Configuration A, The Primarv Decision 
Maker and its Quantized Consultant. 


The problem of the PDM is 
1. to design the quantization rule @ i.e. to specify the set of N points 
-O =X, SX, 5... 5 KY < W 
that defines the quantizer intervals, and 
2. to design the decision rule y, 


¥2? Yq ¥ ¥2 7 (0,1) (3.2) 


in order to minimize the decision cost. 
2. Problem Analysis 

Our approach is as follows. We first design the optimum Bayes decision rule 
given a set of quantizer parameters. Next, the average cost is expressed as a function of 
these parameters. We then minimize the average cost with respect to them. 

a. The Optimum PDM Given Some Quantization Rule 4@ 

We have shown in Chapter [ that, to within an additive and a 

multiplicative positive constant the average cost is given by [6] 


R=CP.-P (3.3) 


d 


where C is the ratio of error costs and P, and P,, are the probability of false alarm and 


probability of detection respectively. The PDM receives a quantized level ee Q 
He will make his decision on the basis of his own observation y, and Shien The 


performance of the the primary decision maker, given some quantization rule @, 1s 
given by the following lemma. 

Lemma 3.1 

The probability of detection and probability of false alarm of the Primary Decision 


Maker are given by: 


P > oe fly,,y, /H, ) dy, dy (3.4) 
d iy x eZ, eel a 2 


and 


Sy 


NO xX. 
Ei -> niet {(¥,.¥> /H, ) dy, dy Ga 
f ys Jy ye Zl De SO ae eg 


where Zi; is the decision region Z, given that Y,ElX,,X; +1]: 
Proof: 
The proof is given in Appendix (A). 
The decision rule of the Primary Decision Maker is given by Theorem 3.1. 
Theorem 3.1 
Given y, P and y, the decision rule of the Primary Decision Maker of Figure 3.1 is 
1. deterministic 


ose oe (nL) (3.6) 


2. a likelihood ratio test 


1 ifA(y,) 2 0, 
u = { te j= L2josN (3.7) 


0 if A (¥5 — 0, (Y5 ) 


where A (y, )=fly, /H, )/f¥>/Hy ) 
the threshold function 0; (y, ) is given by 


U2 


Ne 
ie "Ay /¥pHp) dy, 
O.(y,)= Ct —————__ j=l... (3.8) 


S x, 
J x AY ,/¥q0H)) dy, 
Proof 


We first insert (3.4) and (3.5) into (3.3). Each term of the detection cost (3.3) is then 


given by 


ne 


R, —. Zz, i LC fly, .¥o/H,) fly cyo/ ica (3.9) 
“ J J oF 


To make R; in (3.9 ) negative an optimum decision rule assigns y, to Z, if 
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X. ; é 
C {it ky, V9 /Hy ) dy, : fit fy, 5 /H, ) dy, = 0 j=1,2,..N (3.10) 
“a j 


otherwise it will assign y, to Zp . 
Applying Bayes rule and rearranging terms, decision rule (3.10) can be 


written as 


X. 
bes * Ky, /¥sH,) dy, 
A (y,) 2 C —L——__., j= 12,...N (3.11) 


me ty, /y5,H,) dy, 
5 


which completes the proof. 


b. Optimum Quantization of Y | 

According to Theorem 3.1, the decision rule of the PDM 1s a likelihood 
ratio test with data dependent threshold. The threshold depends on the choice of X's. 
To find an optimum solution for the X, "s is not any easier than that of the DD 
problem. Recall that for the DD problem optimum solutions are possible only for the 
case of conditionally independent observations. Only suboptimal solutions are possible 
for the case of correlated observations. We will not expect more for the QD problem. 
Let us consider each case separately. 

3. Conditionally Independent Observations 


Under the assumption of conditionally independent observations, 1.e. 
Poy yo) = fy,/ A) | (3.12) 


the decision rule of the Primary Decision Maker can be simplified. This decision rule 1s 
given by the following corollary of Theorem 3.1. 

Corollary 1 

Assuming conditionally independent sensor observations, and given ane and y.,, the 
decision rule of the Primary Decision Maker of Figure 3.1 is | 


1. deterministic 


Yo? Yigk %2 7(0,1) (3.13) 
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2. a likelihood ratio test 
1 ifA = ©. 
ee (¥) lS =12 (3.14) 
: 0 ifA(y,)< 0, 


3. the threshold 0; is given by 


X. 
J x AYy/Hg) dy, 
SS ——————_—— j= Le (3.15) 


J X. 
7 : 
j x. fly, /H,) dy; 


Proof 
By applying condition (3.12) in the threshold equation (3.8 ) one obtains (3.15) which 
completes the proof. 

Let us denote the conditional probability of detection and the conditional 
probability of false alarm of the PDM given that the j,, quantization level of y, 1s 


received by P dj and Pe. Let ¥ be the set of all points y, for which 
A, (y,) 2 0, (3.16) 


Then P dj and Pr can be written as 


Pai aa ) va fly, iH, ) dy, (3.17) 
j 
and 
Pe a j we ly, [Hy ) dy, (3.18) 


J 


Equations (3.4) and (3.5) are now given by 
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x. 
Pg -> fo Ry (Ay) dyy Jy, fy. /Fy ) dy, (3.19) 
: J 


and 


Ps 


»« 
x GPa Pg) or Sf Hy) 839 (3.20) 


Substituting (3.19) and (3.20) in (3.3), then differentiating R with respect to X, 
{j= 2,3,...N} will yield necessary conditions for optimality of the set of N equations. 


C [f pir 5 /Hy )dy, = {(X, Y5 [Hp )dy, - 
k k-l 


[ f Xe y,/H, )dy, - ty - Y, /H,)dy, ]=0 ,k=2,3,...,N. (3.21) 


Applying Bayes rule and rearranging terms, (3.21 ) can be written in the following way. 


| J {(y5/Hg,) dy, 2 § ip K(v5/ Hp) dy, 
k-| 


A(X, )=C — 


i K= CIN (S322) 
te fie Ein) ces - i I(y,/H,) dy, 
Pr. ie 


The set of N-1 necessary conditions (3.22) are general for any statistics of y, . For the 
special case when A (y, ) is monotonic in y, , let qT be the value of y, for which 
0, =A, (T;) hile 2. (3223) 


So qT, is given by 


oe 
§)7 ty / Hy) dy, 
a fe eN (3.24) 


a T+ 
j x, fly, /H,) dy, 
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For this case of monotonic A (y, ) the set of necessary conditions for optimality (3.22) 


can be written as 


co co 
en f(y 5/9) dy, . J 7 l¥2/ Ho) 3 
SC k= 2 Joe (3.25) 
CO co 
) 7 {¥2/H)) dy - ie f(y,/H,) dy, 
k k-] 
Equivalently we can write (3.25) in the form 
T 
J _"fly2/Hy) dy, 
A(X, ) = C —&—————_s>#k=2,3,...,N. (3.26) 
j_'flyy/H)) dy, 
k 
P, and P, in this case are given by 
NOX, oO 
P, >) fi" My, /Hy) dy, J. fly, Hy ) dy, (3.27) 
i 1 
and 
Ns gone oO 
a - ] } r / 
Es -3 fx 7a Hig) dy $5. Ml¥2 Hy ) dyp (3.28) 


Equations (3.24) and (3.26) are only necessary conditions for optimality for 
monotonic likelihood ratio. They correspond to minima if the Hessian matrix 
[.0°R/8X;8X]] is positive definite. All solutions must be checked for the global minima. 


4. Solution of the Primary Decision Maker Problem with Independent Sensor 
Observations and Monotonic Likelihood Ratio 


The following theorem summarizes the above solution of the PDM with 


independent sensor observations and monotonic likelihood ratio. 
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Theorem 3.2 
The decision rule of the Primary Decision Maker with a Quantized Consultant (for 
independent sensor observations and monotonic likelihood ratio) is; 


1. deterministic 
%2° Vig k ¥2 7 (0,1) (3.29) 


2. a likelihood ratio test 
l if A (y, ) Ze. 
u = we 12,25 (2.39) 
° 0 ifA(y,) < 9 


where A (y, )= fly, /H, )/fly5/Hg ) 
3. the threshold function © (y, ) is given by 


x, 
J 27 Aly,/Hp) dy, 
FSS a 10, (3.31) 


X. 
J. Ay JH) dy, 
s 


The optimum set of quantizer interval end points must satisfy the set (3.26 ), where 
T,s are given by (3.24). All possible solutions must be checked for the global 
minimum cost. 
5. The Case of Correlated Observations 

We now move to a more realistic situation by removing the condition of 
independent sensor observations. In many radar and sonar problems noise in nearby 
sensors is likely to be correlated. As we mentioned before the decision rules (3.11) are 
likelihood ratio tests with data dependent thresholds. It is impossible to come with 
their optimum functional expressions [4.] A suboptimal solution for the case of 
correlated observations is to use likelihood ratio tests with constant thresholds as local 
decision rules. These constant thresholds for y, are the values of y, for which the 


inequality (3.11 ) is an equality. i.e.; 
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X 
J on MAY{/ TH) dy, | 
A(T, ) = C —*+—————__ k= 1,2,...,.N . (3.32) 
(3 et fy ,/T, 5H) dy, 
Ay 


In terms of these thresholds T, ‘s and the quantizer points X, ‘s one can write 
expressions for the probability of detection and the probability of false alarm in the 
form of (3.4) and (3.5). Substituting for P, and P, in (3.3) and differentiating R with 
respect to X, for k=2,3,...N yields the following set of necessary conditions for the 


case of monotonic A, (y, ): 


ili 
k-l 
j T {l¥2/X,,H) dy, 
SOG) = Cee k=2.3,.N (3.33) 
K-ly /X, H,) d 
Jy ly,/X,,H)) dy, 


The set of equations in (3.32) and (3.33) constitute 2N-1 equations that specify the 
quantizer interval end points {X, } for y, and the thresholds {T,} for y, . 


C. TEAM DECISION OF TWO QUANTIZERS AND A FUSION CENTER 
In this section we will consider the problem of making a global decision based on 
two quantized observations. 
1. Formulation of the QD problem 
For the structure of Figure 3.2, y, is quantized into N levels by the 


quantization rule a, 
Ce pa Nag (3.34) 
and y, is quantized into M levels by the quantization rule @,, 
G,:Y,> Y 24 : (3.35) 
The quantized values i and Yoq are sent to the fusion center which must decide which 


state of the phenomena is true. It is required to design the quantization rules @, and 


a, and the decision rule ¥ 
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FOX Yo, (0/1) (3.36) 


to minimize the global cost. 
2. Problem Analysis and the QD Algorithm 
The observation space of the fusion center contains NM points to be divided 


2NM fusion methods, checking all 


into two decision regions. Since there are as many as 
of them will consume a very long time even for small values of N and M. A suboptimal 
solution is to approximate the threshold equation of the corresponding CD problem by 
a piecewise curve in the y, y, plane. This is illustrated in Figure 3.3. 

The figure shows a schematic diagram of a CD threshold curve and its 
staircase approximation. The approximate curve consists of segments of straight lines 
connected together. The coordinates of the connecting points will play the role of the 
interval end points of the quantizers. Let us first write an expression for P, and P, in 
terms of these point coordinates. If this expression of the cost is minimized with 
respect to each coordinate there will be as many equations as the number of 
coordinates. Solving these equations simultaneously yields the quantizer parameters. 
This is the core of the QD algorithm which is summarized as follows: 


I. Derive the threshold equation of the CD system. 


A(¥, 73) =C (3.37) 


2. Approximate the threshold equation by a stepwise curve satisfving the N and M 
constraints. 


3. Write an expression for the cost in terms of the curve parameters. 
4. Minimize the average cost with respect to the curve parameters. 
Let us illustrate how the algorithm works for the case of detection of a known 
signal in gaussian noise. 
3. An Example: The Known Signal in Gaussian Noise 


Consider Figure 3.2 when y, and y, are given by 


Hy: y, = 0. 
Hy, 2 lms (S035) a, 


I 
$> 
+4- 
jo) 
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Figure 3.2 Configuration B. The Team of Two 
Quantuzers and a Fusion Center. 
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Figure 3.3 Quantized Threshold Curve. 
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eZ 


where the a,’s are positive constants and WN = [n, n, ]' is a gaussian random vector of 


zero vector mean with covariance matrix: 


K = f "i (3.39) 
p |i 


It is required to design the N-level and the M-level quantizers Q, and Q, 
and the decision rule y where . 


~~ 


Yaa on (3.40) 


to minimize the average decision cost. 
Procedure following the QD algorithm 
The threshold equation of the CD problem has been shown in Chapter II to have the 


form: 


(a, -p a, )y, +(a, +p a, jy, =(a,? +a,7 -p a, a, )/2+(1-p7 Nlog(C). (3.41) 


1. The CD curve is a straight line in the y, y, plane. 


2. Possible stepwise approximations for the threshold equation are shown in 
Figure 3.4. We notice that in Figure 3.4 a and c the two quantizers have the 
same number of quantizer levels. While in Figure 3.4 b and d one quantizer has 
one more level than the other. From Chapter II, we can expect thatuiae 
constant C will decide the superiority of a or c and of b or d. We shall 
consider optimum parameters of Figure 3.4 a and b. Similar treatment can be 
considered for Figure 3.4 c andd. In Figure 3.4 a the point X, =-© while T, 
is finite. In Figure 3.4b X,=-© and T,=%. 


3. The probability of detection of the decision rule of Figure 3.4 a is given by 


Ne ee 
eee -> Ix Jp Ap ¥2 1H, ) dy, dy, (3.42) 
and P, is given by 


\! x CO 
p=y i+] ry, /H, ) dy, dy, . 3.43 
fae Is J yy | 9 ) dy, dy, (3.43) 
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For the detection rule of Figure 3.4 b P, is given by: 


cit eS 
P 75 X. Py 72 /H, ) dy, dy, (3.44) 
i i 


l 


and P, is given by 


P mats /H, ) dy, dy (3.45) 
f -2£ vie2 0 ce . 
: cae i 


4. Necessary conditions for optimality of parameters of the curve in Figure 3.4 a 
are: 


x 
faa fonaeert.) dv. 
A (T,) = C = i= 12,..,N (3.46) 
(i? Wy, /T.,H) dy, 
X, 


and 


T. 
J! fy,/X,,H,) dy, 
A(X,) = C —L-——____—. i= 23....,N. (3.47) 
ae 
) ; {(y,/%;,H,) dy, 


1 


For Figure 3.4 b ,the optimality conditions are 


X. 
f ida ay te iig)) dy, 


maT, ) = C ; i Geen (3.48) 
ae f(y, /T, Hy) dy, 
1 


and 
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a 
le f(y5/X,,Hp) dy, 

A(X) = C1 £23. (3.49) 
i i{y,/X,,H,) dy, 


1 


The last two equations are exactly the same as the necessary conditions for 
optimizing detection using a Primary Decision Maker and its quantized second opinion 
for the same signals in gaussian noise. Recall that the information available at the 
PDM is more complete than that available at the fusion center of two quantized 
observations. Yet the two problems have the same solution. This is a proof of the 
following lemma. 

Lemma 3.2 

Optimum detection of known signal in gaussian noise using two quantized observations 
of N and N+1 levels is equivalent to optimum detection using the first quantized 
observation and the second continuous observation. 

Lemma 3.2 is applicable to anv case with a monotonic likelihood ratio. This can be 
easily proved by writing the necessary conditions of optimality for the two 
configurations. A special case of Lemma 3.2 is that of N = 2. It corresponds to the 
tandem configuration of two detectors in a Distributed Detection Network (DDN) [10]. 
The “downstream” detector (decision maker) makes its decision based on its own 


observation and the “upstream” detector’s decision. 


D. NUMERICAL SOLUTION FOR THE SYSTEM PARAMETERS 

It is of interest to compare the four sets of equations {(3.24 ),(3.26)}, 
{(3.32),(3.33)}, {(3.46 ),(3.47)} and {(3.48),(3.49)} with that of Lloyd and Max [18,23] for 
minimum distortion quantizer parameters. 

Max’s trial and error algorithm to solve this set of nonlinear equations can be 
used. However Max’s algorithm is very time consuming [24]. We have used instead 


the method of successive substitutions with an initial guess satisfying 

< < 
X, SX, 5.35 XQ (3.50) 
and put the equations in the form 


Z=G(Z) (3.51) 
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Figure 3.4 Possible Approximations of the Threshold Equation. 
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The kth iteration is then given by 


Zee (Zee) ro (3.52) 


We will devote the next chapter to solving some numerical examples using this method. 


E. SUMMARY 

In this chapter the method of detection using quantized sensor observations has 
been introduced. This method, referred to by QD, can have significant performance 
improvement compared to the distributed detection algorithm (DD) with only 
marginally more demand on the communication channels. The QD algorithm involves 
approximating the CD threshold hyperplane by a stepwise hyperplane that can be 
spanned with the quantized data and that minimizes the detection cost. 

Also the equivalence between two detection configurations, one with tandem 


connection and the other with hierarchical structure, has been shown. 
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IV. NUMERICAL RESULTS 


In this Chapter some examples are solved numerically using the QD algorithm. 
First the detection of known signals in gaussian noise is considered. Next detection of 
signals with exponential distribution is considered. Finally, the algorithm will be 
applied to differentiating between gaussian signals with different variances. 


A. KNOWN SIGNAL IN GAUSSIAN NOISE 
Again consider Figure 3.2 when y, and y, are given by 


° Nn. 
I 


pee te 1 12 (4.1) 


4 


Hy : 
H, : 


With a, = 4 and a= 2. The noise vector 
N= (n,n, | (4.2) 


is of Zero vector mean and with covariance matrix given by: 


K = k d (4.3) 
p 1 


where p is given by 
Pee e {n,n} . (4.4) 


It is required to: 


1. Design the primary decision maker PDM and its N-level Quantizer to minimize 
the average decision cost. We have designated this structure configuration A. 


2. Design the N-level quantizers Q, and Q, and the decision rule u, to minimize 
the average decision cost. We have designated this structure configuration B. 


3. Compare the performance of the two configurations and that of the completely 
centralized system. 


Following the algorithm we have: 


1. The threshold equation for the CD problem given by, 
is 


(a, -p a, )y, +(a, -p a, )y, =(a,? +a,” -2p a, a,)/2+(1-p? )log(C) (4.5) 


a straight line in the y, y, plane. 


2. Figures 3.4 a and 3.4 b show the decision regions for configuration A and 
configuration B respectively. 


a Probability of detection P,, and probability of false alarm P, of PDM are 
given by: 


a 72 Cie exp(-y,7 (2) Tenle e a eel : dy, (4.6) 


and 





Ne l py, 
P, = . iF] exp(-y,* /2) erfc LS diva 4.7 


Also P,, and P,. of configuration B are given by: 


EN. l T, -a,-p(v, -a, ) 
Se x iT exp(-y,* /2) erfe J +—+—-- 1-6 dy 4.8 
2d Aira een?) { i y, (48) 


5 T,- 
oh - exp(-y,“ /2)  erfe oo. bay (4.9) 


4. For configuration A equations of the quantizer interval end points and 
corresponding PDM’s thresholds for the gaussian case are given by 


J/ (1-p?) 


A(X, ) = cites Adie) ____ 
a: Ty.1749 -P (Ay -4y) 2? -p (X K-41) - p(X, “ay eh 
(1p?) verte RS Jp 


K=2,3, N 


and 
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T, X, -p T, 
Egic Ais ek the oy a 
A(T, ) = ee ie) Ak=1,2, N. (4.11) 


k x -a, -p (T,-a,) -a,- p(T, -a, ) 
Reker lee 2 k 2 
rey vp) pe a J (1-p?) j 


For configuration B the quantizer end point intervals X’s and T’s are given by: 
T, ,-p X -p X, 
erfc —s - erfc a 5} 


ed ar ae eo eae 
fi Bolo gan Kk. 7. k om | 
tee nf (ep) asf a J (Ip) } 





k=2,3, N (4.12) 


and 


“p -p T, 
erfc Aya] Ai he He) APT k 7 

e J (1p? 
Ve v (lp) K=2,3, N. (4.13) 
Ay +1 74) °P —) er: Ay 7 a - P CT, -ay . 


J (1-p7) J/ (1-p7) 


We have solved the system of equations of the two configurations using the 
method of successive substitution for N = 2, 3, 4,5 and 6. Figure 4.1 shows the 
receiver operating characteristics ROC for the two configurations for p=0, for different 
values of N. The ROC for the CD system is also shown. The effect of p is illustrated 
in Figure 4.2. The figure shows ROC curves for Configuration A for different values of 
N and for p=0 and 0.25. Figure 4.3 shows the average cost of Configuration B and 
CD vs. C, for different values of N. The relation between the cost of detection for 
Configuration B vs. the number of quantization levels is shown in Figure 4.4 . The 
figure shows the exponential decay of the detection cost as the amount of information 
available at the fusion center increases. 

The following results are noted from the curves. 


1. Configuration A has better ROC curves than Configuration B. The 
performance difference is large for N = 2 but gets smaller as N increases. 


2. Both performances converge to that of the CD in a uniform manner. 


3. As the correlation coefficient increases the performance difference decreases. 
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Figure 4.2 ROC Curves for Configuration A for p =0 and 0.25. 
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Figure 4.3 Average Detection Cost for Configuration B and CD. 
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Figure 4.4 Average detection cost for Configuration B vs. N. 
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4. As N increases the average detection cost gets smaller and tends to that of the 
CD. Since the number N reflects the mutual information between the input and 
the output of the quantizers, the relation between the performance degradation 
and information delivered to the fusion center is strong. 


B. SIGNALS WITH EXPONENTIAL DISTRIBUTIONS 
Consider again Figure 3.2. Let y, and y, have the following distributions: 


_ Hy: fly, ) = ho exp(- ho y; ) (4.14) 
and 


Ei fly, ) = A, exp(-A, ¥, ) ta ti2 (4.15) 
and assume that A, is less than A, . It is required to design the quantizers and fusion 
rule that minimize the average decision cost. 

Following the QD algorithm we have: 


1. The CD threshold equation is given by 


where C, is given by 


a a 
i eee, (4.17) 
hy - Ay 


The CD threshold equation is a straight line in the first quadrant. 


2. Figure 4.5 shows possible approximations of the threshold equation. For N=2, 
the symmetry suggests equal detector thresholds. For N 2 3 let us fix X, and 


oN LO ZEl 0. 
3. The probability of detection and probability of false alarm P, and P, are given 
by 
es -> [ exp(-A, X, )- exp(-A, X,,, )] exp(-A, T; ) (4.18) 
and 


82 


N 


Pe = ) [ exp(-Ay X; )- exp(-Ag X;4, ) ] exp(-Ag T; ) (4.19) 
i=1 


4. Writing an expression of the average cost in P, and P, as before and minimizing 
with respect to X, ae Oy IN adams, k= 1,2,.. N-l one obtains the set of 
equations 


exp(-A,) X, )-exp(-Ag X_, ) 


k= 1,2,... N-1 (4.20) 
exp(-A, X, )-exp(-A, X, 4 ) 


d 
exp{(A, -A, )T, }= 7 C 
] 


and 


exp(-A, T, )-exp(-Ag T. 4, ) 


k=2,3,..N. (4.21) 
exp(-A, T, )-exp(- Ay Ty 41) 


. he 
exp{(A, -A, )X, } = x C 
] 


This set of equations Have been solved by the method of successive 
substitutions for Ao = 2, A, =], and for N=2,3,4,5 and 6. A FORTRAN 
program to calculate the quantizer parameters is given in Appendix D. 
Figure 4.6 shows ROC curves for the quantized as well as the CD systems. The 
average detection cost is shown in Figure 4.7. 
We note the following: 


1. The largest performance improvement occurs when we switch from N = 2 toN 
= 3(1.e. only less than one more information bit per detection). 


2. The performance curves { ROC(N) } and { RCN) } converge uniformally to the 
performance of CD 


C. GAUSSIAN SIGNALS WITH DIFFERENT VARIANCE 
Consider again the structure of Figure 3.2. Let sensor observations y, and y, be 
independent, identically distributed gaussian random variables of zero mean. However, 
under Hy, Var(y, )= Che 
and 
under H,, Var(y, )= Sic ji=1,2. For specificity, let 
ee lands, =~ 2. 
Quantized sensor observations are sent to the fusion center to decide which of the 
hypothesis is true. It is required to design the quantizers Q, and Q, as well as the 


fusion rule to minimize the average decision cost. 
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Figure 4.6 ROC curves for Exponential Signals. 
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Following the QD algorithm we have: 


1. The CD system decision rule is a likelihood ratio test. The CD detector declares 
H, is true if 


y,? +y,* < (1/2)log(e,? /(6,7 Cy? -6,7 y" (4.22) 
otherwise it will declare Hp is true. The threshold equation is the circle 

Waa yos = Ro (4.23) 
where Ree is the right hand side of inequality (4.22 ). 


2. Possible approximations of the CD threshold equation are shown in Figure 4.8 . 


Figure 4.8 a corresponds to 3-level quantizers. The corresponding probability of 
detection and probability of false alarm are given by; 


Py (3)=[erf{-X/o,)]}° (4.24) 
and 
P, (3)=[erf(-X/6,)]? (4.25) 


Where y, and y, are subdivided by the points X and -X. For the 5-level 
quantization approximation of Figure 4.8 b, the probability of detection and 
probability of false alarm are given by 


P, (5)= erf(X, /S, ) {2erf(X, /6, )- erf(X, /6, )} (4.26) 
and 
P, (5)= erf(X, [Gq ) {2erf{X, [Gq ) -erf(X, [Gq )} (4.27) 


Where X, , X3 ,-X, and -X, define the the quantization intervals of both y, 
and y, . 


4. Inserting P, (3) and P, (3) into R in (3.3 ) and minimizing R with respect to X 
gives 
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o, erf{(X /d, ) 


A(X)=C 
Sy erf{(X /o, ) 


(4.28) 


Also inserting P,(5) and P{5) into (3.3 ) and minimizing R with respect to X, 
and X, gives; 


ee erf(X, /Op ) 
A(X, ) C 5, eci{X, /6, ) io, ) (4.29) 


AG) - co 2 (4.30) 
Oy erf{(X, /o, ) - erf{X; /o, ) 


Solution of these implicit equations in the quantizer parameters can be carried 
out by the method of successive substitution. The FORTRAN program to calculate 
them for any value of 6, and o, is given in Appendix F. 

Figure 4.9 shows the average detection cost vs. C for 3-level and 5-level 
quantizer systems. Detection cost of CD is also shown. The figure shows that the 
detection cost decreases dramatically using 5-level quantizers in comparison to 3-level 
quantizers. The cost of the CD svstem is only slightly lower than that of the 5-level 
quantizers. 

Similar procedures can be carried out for the case of correlated observations. The CD 
curve in this case is an ellipse with principle axes passing through the origin. It can be 


approximated in a similar way as the circle. 


D. CONCLUSION 

The above examples show the uniform convergence of the Quantized Detection 
Algorithm to the Centralized Detection Algorithm. The Distributed Detection 
Algorithm is a special case of QD. It follows that Quantized Detection is an efficient 


utilization of bandlimited communication channels. 
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b) Approximation with 5 Levels 


Figure 4.8 Possible Approximation Of the Threshold Equation. 
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Figure 4.9 Average Detection Cost. 
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V. THE CASE OF VECTOR OBSERVATIONS 


A. INTRODUCTION 

In the previous two chapters the QD problem for the case of scalar sensor 
observations was solved. It is now time to extend the QD algorithm to the case where 
each local observation is a vector Y, . The QD algorithm can be applied as long as the 
corresponding sufficient statistic for the centralized detection problem can be divided 
into local statistics to be quantized. Let us consider the gaussian case and put it in the 


previous framework. 


B. QUANTIZED DETECTION WITH VECTOR OBSERVATIONS 


For the structure of Figure 5.1 the observations at locations | and 2 are given by 


=A +N, j= 1,2. (5.1) 


y = - (5.2) 


The noise vector W , given by 


I= 
i 
I= [= 
io 
in 


is multivariate gaussian with zero vector mean and covariance 


k= (5.4) 


91 


p 


e PHENOMENA 


IZ Sh 


| ] 


= oe 
Q2 


liq l2q 
Fusion _ ee 


ud 


Figure 5.1 Vector @Gservations. 
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R, ,.R.and R.. are the covariance matrices of the noises at locations | and 2 and their 


ee = 12 
common covariance matrix. The signal vector A is given by 


I> 


A= (5.5) 


> 
N 


The CD system decides that ¥ belongs to Z, if [6] 
exp { (-1/2)[ (¥ -A)'R! (¥-A)-Y'R' YJ} 2. (5.6) 


The CD threshold equation can be written in the form 


AR? ¥ = log(C) -(1/2) A’'B" A. (5.7) 
Using the block matrix inversion lemma [25], (5.7 ) can be written in the form 
GY, +8 Y, = log(C)-(1/2) AR A. (5.8) 
In (5.8 ) a and B are given by 
BA: -| - 
a =A’, (8, -By. 2.” Bo, ) 
“A’, (By -By, By By” Bay By (5.9) 
and 
eA - -| -! 
B =-A’, (8, -B). 8. Ba) ) By Bs 
+A‘, (By Bp) By Bin) (5.10) 
Denoting 
L=ay, (5.11) 
and 
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L= B ¥, | (5.12) 
(5.8 ) becomes 
L, + L, = log(C)-(1/2)A'R' A (5.13) 


where |, and |, are bivariate gaussian with zero vector mean under hypothesis Hp . 


Under hypothesis H, their vector mean is 


i —! 
E } 3 (5.14) 


Covariance of |, and 1, is given by 


aR, @ p 
Covi) is (5.15) 


In (5.15) p is given by 


p= 08, PY | Bos W(paen pry (5.16) 


The distributed signal processing is to form local linear combinations |, and |, , then 


quantize them as before. This processing is also shown in Figure 5.1. 


C. SUMMARY 

In this Chapter it is shown that the QD algorithm can be extended to the case of 
sensor vector observations. An application is the case of high quality local area 
communication and lower quality long distance communications. In this case sensor 
observations in local areas are gathered at a local processor to form the local sufficient 
statistics. Quantized local statistics are then sent to the global far away processor for 


fusion. 
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VI. OPTIMUM ESTIMATION USING QUANTIZED SENSOR 
OBSERVATIONS 


A. INTRODUCTION 

In the previous part of this thesis there are situations in which a group of 
observers make local decisions that, taken in combination determine the overall 
performance of a system. The observers may or may not be interconnected. However, 
even when they are, for a variety of considerations such as limitations on 
communications bandwidth, transmitter power,security, or perhaps the very nature of 
the observers themselves, only decisions may be interchanged between them and not all 
the observations upon which their decisions are based [1,5,26-33]. 

Another case of interest concerns the encoding of high resolution measurements 
for transmission between observers using a small number of bits. Here a remote 
observer must decide which of N possible discrete values best represents his 
observation. A second observer is to combine his local observations with the discrete 
data from the first in an optimum manner. In this chapter we consider the problem of 
regeneration of a remote sensor observation using its quantized representation and a 
local observation. The design of the quantizer at the remote sensor location and the 
Optimum linear estimator to combine the quantized data with the local observation to 
minimize the expected mean square estimation error will be considered. Generalization 


of the results to the vector case is also shown. 


B. THE LINEAR MINIMUM MEAN-SQUARE ESTIMATE OF Y , 


Consider the structure of Figure 6.1 in which the observation y, is quantized 


into Mig by a quantization rule y 
Viv oy (6.1) 


The quantized data Vig is sent to sensor S, site. 


7 


The linear minimum mean square estimate of the observation y, from via and y., 1s 


shown in (Appendix F) to be 


A 


6 | 
“eS G p Ry +P (np vs] [(n -H? p* ) (6.2) 
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Figure 6.1 Estumation Using Quantized Sensor Observations. 
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where we have assumed that y, and y, are random variables with zero mean and 


variances 
E{y2,} = 0% i=1,2 (6.3) 
and correlation 
p =Ety, y, }/o, 5, (6.4) 


_ The scalar quantities n and }t are parameters of the quantizer and are given by 


N 
n= (¥ P,Q W/o," (6.5) 
and 
y) 2 
B= (92, & Cy 6," (6.6) 


N is the number of quantization levels and C, and P, are given by 


X 

a () x ly, ) dy, )/P, (6.7) 
and 

Ay + | ) 

P= LET Ry, ) dy, (6.8) 


sek 


Q, is the k, quantization value and Q,eé[.X,,X,,,]- In (6.7) and (6.8), X, 
”K= 1,2,....N, are the quantization interval end points, with 
wee and xX, = 2. 


It is required to design a quantizer that minimizes the mean square estimation error. 


oe) 


The expected mean square error is given by [34] 


EU(Y, “Vig? = 6,2 -E(y, PEE FYE yy} (6.9) 
where Y isthe vector 

Y= [4 Y2)- (6.10) 
Equation (6.9 ) can be wnitten in the form (Appendix D) 

E{e? }=0,7 (1-p* X1-@ )/(1-p? @ ) (6.11) 
where @ ts given by: 

w =p? /n. (6.12) 


A plot of E{e*}/o,* vs. @ is shown in Figure 6.2 for p?=0, 0.25, 0.5, 0.75. The figure 
shows that the mean square error is decreasing with @. Recall that the criterion 1s to 
minimize the mean square error. 


Equivalently the problem now is to maxinuze @ over all quantization rules where 


(Pe Pe Se Oe Cala 


l 
o=— (6.13) 
sy ae 
re Py 
Appling the Cauchy Inequality [35] to the numerator vields 
N! 5 f > : r ae : 
y PQ Cy) oo. P,Q 1, paca» (6.14) 
with equality if and onlv if 
Ore <. (6.15) 
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Figure 6.2. Relative Mean Square Estimation Error vs. 0. 
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Therefore 


i 
2 
® < se Peon (6.16) 


gives an upper bound of w. Equation (6.15) says we maximize @ , and thus minimize 
E{e?} by making the quantization level Q, equal to the conditional mean of y, given 
that y, hes in the kth quantization interval. This is one of the conditions 
characterizing the classical Lloyd-Max quantizer [18,23.] There remains the problem of 
how to pick X, ,k=1,2,...N, so that the upper bound of @ in (6.16) is maximum. 
Notice that the upper bound of @ is n. Therefore, the optimum quantizer will be a 
Lloyd-Max quantizer if we prove that maximizing n over all choices of the set of 


points {X,}, k= 1,2,...N, is equivalent to minimizing the distortion E((¥)-¥yq)"}- Since 


E((Yp Vig} = 9)7-26,-N+6,7 0 (6.17) 
7 cra 1-1) 


then maximizing n will minimize the distortion EY, %,4)) and vice versa. Since the 
Llovd-Max quantizer is the optimum quantizer for minimum distortion it follows that 
it is also optimum for our problem. Accordingly choose A, ‘s such that [23,13], (see 


also Appendix G) 


_— 
X, = we A= 1,2,...N. (6.18) 


Equations (6.15) and (6.18) along with (6.7) completely design the quantizer [23,18]. 
Parameters of the Lloyd-Max quantizer can be calculated efficiently using the method 
of successive substitution (Appendix G). Values of Efe? jis? vs. N are listed in Table 
2 for p =0, 0.25, 0.5, 0.75. The table shows the exponential decay of the MMSE as the 
number of quantization levels increases. 

Table 3 shows a comparison between the average number of bits per sample used in 
this system and another method in which the Maximum Output Entropy (MOE) 


Quantizer [36] is used. Huffman coding [37] is assumed for both quantizers. 
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Ie 2 


MINIMUM MEAN SQUARE ERROR VS. THE 
NUMBER OF QUANTIZATION LEVELS 





TABLE 3 


COMPARISON OF THE AVERAGE NUMBER OF BITS IN 
i EyveviSi aN TEE vWiOE SYSTEMS 







Optimum 
Svstem 


oF oy 2.4768 2.8842 
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C. CONCLUSION | 
The trade off between performance and communication is clear from Table 2. 
For p =0.5 the relative MMSE is 0.75 without communication. This corresponds to 
substituting @ = OQ in (6.11 ). The relative MMSE decreases to 0.32 using one 
information bit per sample. The relative MMSE is 0.11 using two bits/sample. It is 
only 0.03 using 3 information bits/sample (N= 8) and is 0.00016 using 7 bits/sample. 
We also notice that for high number of quantization levels the estimation error is 
approximately equal to the the quantization error. This means that the estimator 
depends mainly on y, b for fine quantization. For coarser quantization the estimator 
depends heavily on y, to reduce the MMSE. Table 3 shows that the designed system 
has considerable reduction in the number of bits per sample compared to the MOE 


quantizer svstem. 


D. GENERALIZATION TO THE VECTOR CASE 

In this section we will consider regeneration of a random vector Y, from its 
quantized version Xiq and a correlated continuous scalar y, . As an application 
consider a sensor S, monitoring the activities of N stations. Due to some 
considerations, perhaps of safety nature, only simple sensors can be placed near the 
stations. Because of other considerations, such as limited bandwidth communication 
channels, only quantized sensor measurements can be sent to the monitor. Specific 
examples can be the case of monitoring the states of a target in a far field or the 
positions of N targets in a multitracking problem [38,39]. Another example 1s to 
monitor the radiation levels outside of N nuclear reactors. A third example is 
monitoring the activities of N enemy transmitters. 

Let us design the quantizers at the N sensor sites and the estimation rule at the 
monitor site so as to minimize the mean square error of each component of Y. Let J, 


the sensor observation vector be given by; 
ae ' t 
yy Se Vin J ’ (6.19) 


where Yi; is the j,,, sensor observation ,j= 1,2,...,N. We will assume that components of 


Y are independent.te: 
fly), /¥y, Y=MY; yy) AF 1, t= 1,2,...N. (6.20) 


Under the above conditions, also y,. and Vijq are conditionally independent for 1 = J, so 
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RY; /Zq Yo = NYG Mig V2 ) A= LPN: (6.21) 


The MMS estimate of Y , given Y) : is given by 


XY, = EY) /ZiQ Yn) - (6.22) 
Or 
L/Yite 
E(t > /¥198 5¥3 
= (6.23) 
metus denote the error vector by E , so 
eee, ocx |) (6.24) 


where e. is the error in estimating y. ,i= 1,2,....N. The MMS error covariance matrix is 


E( EE} =E((¥, -, XY, -¥, '). (6.25) 


The trace of the error covariance matrix is given by 


No 2 
trace( E(B E'}) =} _Efe*} (6.26) 
|= 
where 
c a: -Ety); ig De) }. (6.27) 


Minimizing the trace of the covariance matrix in (6.26 ) is accomplished by minimizing 
each summand alone since every summand is nonnegative. Now assuming Linear 
Minimum Mean Square Estimation, the problem of minimizing E ea } implies using 


the Lloyd-Max Quantizer to quantize y,, as was shown previously. 
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In conclusion the Linear Minimum Mean Square Estimate of the observation 
vector Y implies using the Lloyd-Max quantizers at the sensor sites and the same linear 


combining considered in the scaler case at the central processor. 
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VII. SUMMARY, RESULTS, CONCLUSIONS AND SUGGESTIONS FOR 
FUTURE RESEARCHS 


A. SUMMARY 

This Thesis begins bv listing some reasons why Distributed Signal Processing ts 
more practical than Centralized Signal Processing. The status of Distributed Detection, 
an important case of Distributed Signal Processing, and its complexity are reviewed in 
‘Shapter I. 

Chapter II deals with the problem of optimum fusion of local decisions into a 
global decision. The relationship between the optimum fusion rule and the ratio of 
error cost is shown. The dependance of the performance of the optimum fusion rule on 
the correlation coefficients between sensor observations 1s throughly analyzed. For 
higher values of the correlation coefficients the Distributed Detection system 1s shown 
to reduce to the detector of the highest signal-to-noise ratio. 

A compromise technique between Centralized Detection and Distributed 
Detection, Quantized Detection, is suggested in Chapter III. The main issue of that 
chapter ts to control the degree of centralization according to the communications 
channel constraints. The Quantized Detection technique replaces local detectors by 
quantizers and sets a global fusion rule that approximates the centralized decision rule. 
The algorithm matches the other techniques at extreme limits. 

Chapter IV contains some specific applications of the Quantized Detection 
Algorithm for detection problems. A significant performance improvement is attained 
by replacing Distributed Detection with Quantized Detection with three quantization 
levels (one and half information bits per sample vice one information bit per sample). 

Chapter V considers applicabilitv of the Quantized Detection Algorithm to the 
case of vector observations. In this case local sufficient statistics are quantized in the 
same way as before. 

Chapter VI deals with the regeneration of sensor observations from their 
quantized versions and another correlated observation. The local quantizers and the 
Optimum linear data fusion are designed. We arrive at the following results and 


conclusions. 
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B. RESULTS 
1. Detection with Distributed Sensors 
a. Optimum Fusion Rules in Distributed Detection 

The optimum fusion rule depends on the ratio of costs of different types of 
detection errors. For high cost of false alarm relative to the cost of missing the target 
the AND fusion rule is better than the OR fusion rule, and vice versa. 

The performance of the optimum fusion rule depends on the degree of 
correlation between sensors. The performance degrades as the correlation coefficient 
mcreases. The worst performance of the optimum fusion rule is at and above a critical 
value of the correlation coefficient p., - In that region of correlation the best system 
employs only the detector of higher signal-to-noise ratio, ignoring the lower signal-to- 
noise ratio sensor entirely. The performance of the Distributed Detection system 
improves as the signal-to-noise ratio imbalance between sensors increases. However 
there is a large performance difference between the Centralized Detection and the 
Distributed Detection for values of the correlation coefficient above p.. 

Below p._ the performance of the Distributed Detection system improves as 
the correlation coefficient gets smaller. The best performance (lowest detection cost) of 
the Distributed Detection system is achieved at p=-l. Recall that the Centralized 
Detection system has perfect detection at p =-1. This is due to the efficient use of the 
information contained in two observations of positive signals and anticorrelated noise 
samples. 

The large performance difference between Centralized Detection and 
Distributed Detection systems is due to the loss of information in the local detection 
processes. As a remedy to the performance degradation in Distributed Detection we 
have introduced the Quantized Detection Algorithm. 

b. Quantized Detection | 

There is a great improvement in the system performance using Quantized 
Detection with three quantization levels in comparison to Distributed Detection. This 
performance difference between Quantized Detection and Distributed Detection 
decreases as the correlation between sensors increases. 

The Quantized Detection algorithm is applicable to the case of vector 
observations and waveform observations. In those cases, the local sufficient statistics 


are to be quantized at the local processor and transnutted to the central site for fusion. 
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The Quantized Detection algorithm is implemented by the quantizers as 
local processors and a fusion rule, suggested by the Centralized Detection decision rule, 
at the central site. The quantizers used in the Quantized Detection algorithm are 
designed to minimize the detection cost. 

2. Minimum Mean Square Estimation in Distributed Sensor Systems 
Minimum mean square estimation in Distributed Sensor Systems involves the 
classical Lloyd-Max minimum distortion quantizers at the local levels and linear 
processing at the global central level. A faster iterative algorithm to calculate the 
Lloyd-Max quantizer parameters is the method of successive substitution. It also has 


more accurate results than previously reported techniques. 


C. CONCLUSIONS 
We conclude the following: 


1. Global optimization of the Distributed Detection implies picking the fusion rule 
and corresponding local decisions that minimizes the detection cost. 


2. The optimum fusion rule in Decentralized Detection depends on the correlation 
coefficient , the a priori probabilities and the ratio of costs. 


3. For optimum fusion of two local unbalanced decisions there is a particular 
value of p that decides the optimum fusion rule. 


4. For p = p,, OR fusion is better for higher cost of missing target while AND 
fusion is better for higher costs of false alarm. 


5S. Forp > p,, the optimum fusion rule is to ignore the sensor of lower signal-to- 
noise ratio and optimize the decision of the higher signal-to-noise ratio sensor. 


6. The poor performance of Distributed Detection compared to Centralized 
Detection is due to the loss of information at the local levels. 


7. The Quantized Detection system matches the Distributed Detection system and 
the Centralized Detection system for the two extreme conditions of 
quantization. As the number of quantization levels increases the Quantized 
Detection converges to Centralized Detection. 


8. The Quantized Detection algorithm has a tremendous improvement in 
performance over Distributed Detection even with only 3 quantization levels. 


9. The performance difference between Quantized Detection and Distributed 
Detection increases as the correlation of the observations gets smaller. 


10. In case of linear Centralized Detection threshold equations in the observation 
space, Distributed Detection and Centralized Detection are special cases of 
Quantized Detection. 


Ii. The Quantized Detection algorithm can get the maximum allowable 
performance in the presence of communication constraints. 
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The Quantized Detection algorithm can be applied to arbitrary distributions for 
the observations. 


The method of successive substitution is applicable to the design of many types 
of quantizers. It has a simple programming procedure and very accurate results. 


SUGGESTIONS FOR FUTURE RESEARCH 
The following are some areas the Quantized Detection algorithm can extend to: 


Optimum detection using quantized sensor observations for the case of 
unknown signal in noise. 


Detection of M-ary phenomena using quantized sensor data. 
Uulizing the Quantized Detection algorithm over noisy channels. 


Illustration of the relation between the complexity in some suitable units and 
the amount of information delivered to the fusion center. 


Utilizing the Quantized Detection algorithm to meet the Neyman-Pearson 
criterion. 


Extension of Distributed Detection and Quantized Detection to more than two 
sensors with correlated observations and unequal SNR’s. 


Development of general principles for parsing fusion rules given a Centralized 
Detection surface in N-dimensional space. 


Application of the Quantized Detection method to target detection,classification 
and tracking using distributed sensors. 
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APPENDIX A 


PROBABILITY OF DETECTION AND PROBABILITY OF FALSE 
ALARM OF THE PRIMARY DECISION MAKER 


Given that the Primary Decision Maker (PDM) receives Q (the j,, quantization 
level of the Consultant observation y,), and that its own observation is y,, its 
observation space is divided into two decision subspaces Zi; and Zo; Let us denote the 
conditional probability of detection and probability of false alarm given Q by Pa and 
Pe respectively. P dj and Pe are given by: 


Pai = Pr(Declare H, Pile 0: ,H, is true) (A.1) 
and 
Pe = Pr(Declare H, ie = Q ,H, is true). (A.2) 


These can be expressed as: 


and 
5 “§, é a ¥2 Viq = Q Ho Mey, (A.4) 


or equivalently as , 


Pa = | Es J yO Vo /Vyq = Q JH, )dy, dy, (A.5) 
2 )j ] 
and 
Pi a - z,4 yi Vo /Vyq = Q sHy dy, dy, (A.6) 
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But Ky, 5 [Q ,H. ) is given by [40] 


fly, V5 /H, PQ; /H, ) 1 € [X, Keay J 


(A.7) 
QO otherwise 


fly, 199 [Q ,H. = 


where Pr(Q /H, ) is the probability of the j,, quantization level of y, under hypothesis 
H, . It is given by: 


Pr(Q/H,)= f «ty JH, ) dy, j=1,2,..N ,i=0,1 (A.8) 
X, 

The probability of detection and probability of false alarm are now 

P, =F, Pr(Q; /H, )P gi (A.9) 
and 

> | 
gx Pr(Q. (Ho )Pr (A.10) 

Inserting (A.5 and (A.6 ) into (A.9 ) and (A.10 ) yields 


a iu 7 Ray )dy, d (3.4) 


and . 


P= > s, oh @ LY. Mo ) dy, dy, (3.5) 


where Li; is the decision region Z, given that y, €[ X, ora le 
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APPENDIX B 


PROGRAM LISTING TO CALCULATE PARAMETERS OF THE N AND 
THE (N+1)-LEVEL QUANTIZERS 


In this appendix we give a program listing to calculate the parameters of the N 


and the (N+ 1)-level quantizers in configuration A 


ener MERE ee WET AND (Ntl)clevel 


THIS PROGRAM CALCULATES THE OPTIMUM N-LEVEL AND (N+1)-level 
¢ QUANTIZER PARAIE LER S 
OR A SYSTEM OF TWO QUANTIZERS AND THEIR FUSION CENTER. 
C THE RATIO OF COSTS C RANGES FROM Q.1 AND 10 . 
C THE VALUE OF N =2,3,4,5 AND 6 


C THE PROGRAM USES THE MODIFIED METHOD OF SUCCESSIVE SUBSTITUTIONS 
CAAKKAAKAAKAKKAKKARKAKKAKRAKAKARKRAKRKAKKRKAKKAERKAK RAKE RAKKRAKAAKRAKERRKKEKK 


c KEEKRKKRKKRKRKKRAARRRAERAEKRKKKEKREAKKERE 

REAL*8 X(9),T(9),XX(9), TT(9) c,S1, $2, ReeyaAd2 a2) eR, C1(20),R1(2) 

1,AERR,RERR,ERROR,PD,PF,PD1(20,9,6),PF1(20,9 D2 

Pe? Pe pre, k2,Ra020,2,6),Xi1, eee Fl, £3 ‘APD ,APF,A1,A2,AP 
INTEGER K,N,1,P,N1,N2,M,J,MX 
EXTERNAL F1,F2 
COMMON X11,R 

DATA c1/10. .0D0,9.0D0,8.0D0,7.0D0,6.0D0,5.0D0,4.0D0,3.0D0,2.0D0, 

ae. 000 ,. .90D0, .80D0,.70D0, .60D0, .50D0, .40D0, .30D0, .20D0,0.10D0/ 
A1=1.0D0 


SEK nicones 
R1(2)=0.8000D0 
AERR=0 .ODO 


| RERR=0 .00010D0 
CARKKKKKKAKRKKKKAKARKRRRAKKRRRRKRAKRRRKARKRRRARRRRRRRRRKRRKKRKKKE 


C INITIAL GUESS 
¥K(1)=-9.50000D0 
¥%X(2)=-1.9000D0 
¥x(3)=-1.50000D0 
¥x(4)=-0.89000D0 
¥K(5)=00.50000D0 
XxX (6 )=0 .09000D0 
X¥x(7)=00.99000D0 
¥x(85=01.50000D0 
¥xK(9)=2.09000D0 
C TT(1)=4.67000D0 
c TT (2)=3.89000D0 
C TT(3)=2.67000D0 
é TT(4)=2.29000D0 
C TT(5)=1.90000D0 
C TT(6)=1.49000D0 
C TT(7)=1.09000D0 
C TT(8)=00.50000D0 
C TT 9)=0 . 19000D0 
CHAKKKKAKER Pe eee ee ee eH ee I KR 
C THE FOLLOWING INITIAL VALUES OF T'S CORRESPOND TO THE CASE 
C OF CORRELATION COEFFICIENT GREATER THAN A1/A2 
TT(9)=4.67000D0 
TT(8)=3.89000D0 
TT(7)=2.67000D0 
TT (6)=2.29000D0 
TT(5)=1.90000D0 
TT(4)=1.49000D0 
TT(3)=1.09000D0 





ior 


co ei not 

TT(1)=0.19000D0 

DO 500 N=2,2 
WRITE (9,60)N 


onsen eee ee 

C INPUT VALUE(S) OF THR CORRELATION COEFFICIENTS 
R=0, LODO*DFLOAT(Z) 
WRITE(9,61)R 


c DO 200 M=1,20 
M=1 
C=C1(M 
WRITE (9, 61)C 
CRERKAKAKKKAKKAKAAAKKAKAKKARARAAKRAKA AAR RRK 
c INPUT MAXIMUM NUMBER OF ITERATIONS HERE 
MX=250 
K=0 
5 K=K+1 
IF (K .GT. MX) GO TO 10 
Le ak on X(N )=KX(N 
LEK .GI.1) IN) HT 
TF (K .GT. Lpx(NJ =n 
TF UK .GT. Lr jetit 
A= (S1*XX(2)- “axS2 Zt Gls) pape eeee seen al OD0-R**2 )*2. ODGe 
ae S27 Al=Besl“AZ Sees noeen TeOROs BA ODO 
21=(S1*A2-R*S2*A1) /(S1L*S2*DSORT( (1.0D0-R**2)*2.0D0 
T7(1)= (A2*A2/2.0D0+DLOG(C)+DLOG( (DERFC A)- 
i 2.0D0) / (DERFC(A=Al2Z) 2,000) AZ 


T(1)=TT(1 
AS S14TT(2)- Res 20 (2) .ODO-R**2)*2. oo 
nities te R*S1*XX (2) Lee oe RT((1.0D0-R**2)*2.0D0) 
he (A1*A1/2.0D0+DLOG(C)+DLOG((DERFC(A)- 
1 DERE (8), DERFC(A-A21)-DERFC(B- AZ1))))/al 


a iN SEO 20 GO Tog 
DO 15 P=2,N-1 

(Pak P 
S1*X(P eer S1*S2*DSQRT((1.0D0-R**2)*2.0D0) ) 
S2*X (P+1)-R*S1*T(P) ) / (s2xs1*bsoRt ((1. ODO-R**2)%*2.0D0)) 
P)=(A2*A2/2.0D0+DLOG(C)+DLOG( (DERFC(A) - 
1 DERFC(B) B))/(DERFC(A-A12)-DERFC(B-Al12))))/A2 


Er 
A= (ster (rtt)- -R*S2*X(P+1))/(S1*S2*DSORT( (1. OD0=R**2) “ZeGpaee 
B= S2*T(P)- R*S1*X(P+1) )/(S2*S1*DSORT( (1 .ODO=R**~Z Zee 
AACE EL A1l*A1/2.0D0+DLOG(C)+DLOG( (DERFC(A) - 
DERFC(B) i DERFC(A-A21)-DERFC(B-A21))))/Al 





1 
15 CONTINUE 
17. CONTINUE 
TT (N)=(A2*A2/2. ,0D0+DLOG( ¢)+DL0G(DERFE( (Aaa R*TT(N))/ 
LDSORT((1.0D0-R**2)%*2.0D0 
1/DERFC( (XX(N)-Al- SRS (TT(M -A2))/ 
Re RT ( 1 .OD0-R**2)*2.0D0)) 


ery! 1 gp tek Reece Rok teak a oR SOK AIR 


C CHECKING THE ACCURACY 
C INPUT AS eae PRECISION SHER: 


a0 
FR ee -GT. AP).OR. (DABS (X(N) -KxGii Gee 





10 CONTINUE 
al =-10.0D0 
X(N+1)=10.0D0 
APD=0 .0D0 


X11=T(0 
APD=APD+0. 50D0*DCADRE(F1,A,B,AERR,RERR,ERROR, IRE1) 


ke 


O)IR 


IF (IRE1 .NE. 0) WRIT )IR 
oe AERA ,RERR, ERROR, IRE1) 


Eon 

APF=APF+0. SODO*DCADRE (F2,A, 
IF (IRE1] .NE. 0) WRITE(6,60 

81 CONTINUE 

PL ae MD oaPE 


PF1(M,1,N)=APF 
R3 (Mt N)=1 ,QD0+C1(M) *APF APD 
CAKAKKKKKKKKKAKRK ea. KKAKKRKKE KREEKKKRKRRARKKKKKRKRKKKKRKEA 
e QUANTIZER ee 
DO 120 J 


WRITE (9. 62) XC es) 
120 CONTINUE 
C WRITE(9,90) 
C200 CONTINUE 


c pee eae 
C WRITE(9,90 

600 CONTINUE 
ce WRITE(9,90 
e WRITE(9,90 
C WRITE(9,90 

500 CONTINUE 

DO 501 J=1,2 
DO 502 I=1,20 

CRAKAAAKAAKAKAK ARR AARA RAK RARAKAAKRAAARRARERRARRRAKARARARRA RK 
is OUTPUT DETECTION (tT), 

WRITE (10 61)C1(T) 4 Rowe J N) N=2 6) 
CRARKKAAAKRAARKKARAA RAK RARAAKRAK RRR ARR ARRRARRARRRR KARKKERKKKKEK 
ee OUTPUT PROBABILITY OF DETECTION AND “aaa OF 
C FALSE ALARM FOR DIFFERENT VALUES OF N 


WRITE(8,61)( PD1(I,J,N),PF1(1,J,N) ,N=2,6) 
502 CONTINUE 
501 CONTINUE 
60 FORMAT(1X, 14 4(1x F15.8)) 
c90 FORMAT axe NA eke RAKKRAAKKKRKKKAKKKKAKRKRKKRK KK | ) 
61 FORMAT(1X, 1O(1k F6. 2} 
62 FORHAT 1X,2(1X,F15.8 


END 
FUNCTION F1(x) 

REAL*8 X,F1,A1,A2,R,X11,F11,F12 
COMMON X11,R 

AL=1.0D0 

A2=2. 

Fll= SEXP (- (X-AL)**2/2.0D0)/ 
IDSQRT (8. .ODOXDATAN(1.0D0) ) 

=DERFC((X11-A2-(X-Al al 

1(DSQRT (2.0 .OD0*(1.0D0-R**2) ) 
RETURN 


END 

FUNCTION F2(X) 

REAL*8 X,F2,R,X11,F11,F12 

COMMON X11,R 

FLI=DEXP(-X**2/2.0D0) / 
IDSORT (8. LODO*DAEAN (1. ODO) ) 

=DERFC((K11~-X*R) / 

ADSQRT (2. OD0*(1.0D0-R**2)) ) 

F2=F11*F12 

RETURN 

END 
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APPENDIX C 


PROGRAM LISTING TO CALCULATE PARAMETERS OF THE TWO 
QUANTIZERS 


In this appendix we give a program listing to calculate the parameters of the two 


N-level quantizers in Configuration B. 


ORKKKAKKAKAKKAKAKAAAKAAKARKRKAKKAKKAKKAKAAKAKKKKKAKAKRAKKAKKRRKAAKAKKAK 


THIS PROGRAM CALCULATES THE OPTIMUM N-LEVEL QUANTIZER PARAMETERS 
FOR A SYSTEM OF TWO QUANTIZERS AND THEIR FUSION GCEhEeae. 
THE CORRELATION COEFFICIENT IS ASSUMED TO BE LESS THAN Al1/A2. 
FOR THE CORRELATION COEFFICIENT IS GREATER THAN Al/A2 THE 
THE PROGRAM NEEDS SLITE MODIFICATIONS ACCORDING TO THE QD ALGORITHM. 
THE RATIO OF COSTS € RANGES ERGiieGe aNDeo. 
THE VALUE OF N =2,3,4,5 AND 6. 
THE PROGRAM USES THE MODIFIED METHOD OF SUCCESSIVE SUBSTITUTIONS 
KRRARKKAKAKARRKKARRKAKKRAKAKKRRKKKRRKRKRKRRKREKRKRKRKRKKRKRKRRKRKKKRKRAKAKAKKRKRKRAKRKERER 
REAL*8 X(8),1T(8) ,XX(8) ,TT(8) Al Ad acai a T12,T21,AA2 Avs 
1 SER, RERRS ERROR, PO re /PD1(20, =) 6), 'PF1(20,9 O05 "PD2 
L bie PDC, PEC, Glow page AlZ 2c K2 
LXE DCADER, Fl, 2 APD, APF /R3(20, 296) 
INTEGER ney N, ity IER1, IER2, M, P, Ons 
EXTERNAL Fl, F2 
eee ie .0D0,9.0D0,8.0D0,7.0D0,6.0D0,5.0D0,4.0D0, 3.0DG) 2 airs 
D 
11.0D0, .90D0, .80D0, .70D0, .60D0, .50D0, .40D0, .30D0, . 2000 aman 
COMMON ALL, R 


C INPIT SIGNALS HERE 
A1l=4.0D0 


ODO 
C INPIT VARIANCE HERE 
S1=1.0D0 


ANAANANNAAN 


2=1.0D0 . 
CAKAKKKKKAAKKAAAKRKAAKARAKAAKRKARKRKKKKKKKAKKKKKKAKRKKRKKRKKKRAKKARARKK 


C INITIAL GUESS OF THE PARAMETERS 
KX(1)=-0.50000D0 
=0.89000D0 
=01.50000D0 
=2.89000D0 
Brera 
x =4 .09000D 
CRAKKKAR RRR TEASER RAR RRR RIO 
C INITIAL VALUES OF T'S FOR CORRELATION COEFFICIENT GREATER THAN Al1/A2 
TT(1)=-0.67000D0 
TT(2)=0.89000D0 
TT(3 )=01.67000D0 
TT(4)=2.89000D0 
TTVCs)=032 ao en eee 
TT (6 )=4.89000D 
CHAKKKAKARR FO esse Rede kik Rick eRe 
C INITIAL VALUES OF T'S FOR CORRELATION COEFFICIENT LESS THAN Al/A2 
Seas Dera 2) re cssgal M | etokaikki RARER 
AERR=0.0D0 
RERR=0 .00010D0 
DO 500 N=1,5 
DG Lire] 2 
R= DELOAT(1- 1)*0.250D0 
DO 20 M=1,20 
C=C1(M) 
K=0 
5 K=Kt+1l 


rs 
m~* 
DUB Wh 
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C WReue a 60)K,T1,111,12,T22 
IF (K .GT.100) GO TO 10 
DO 25 L=1 
H(L)=22 (1 
mia 
25 CONTINUE 
ry =( heal /2. ee eeeiperrct (4 (1)- R*K(1))/ 
LDSORT((1.0D0-R**2)*2.0D0) 
1/DERFC (T(1)-22- R (&(1) <a) )/ 
1DS RT((1 .OD0=R**2)*2.0D0) ) 


IF (N_-EQ. 1) GO TO 16 
A21=(S1*A2-R Be areca pomens 4 Ono 8 52) 22-000) 
AL2= S2*A1-R*S1*A2)/(S1*S2*DSORT( (1.0D0-R**2)*2.0D0 


15 P=1,N-1 
we (S*K(P) = R*S2*4T(P) (aa eet goe ty .ODO-R**2)*2, ey 
=(S2*X(P+1)-R*S1*T(P (S2*S1*DSORT (1.0D0-R**2)*2.0D0)) 
rT(P)s (A2*R2/2 ebeepu0e ¢)+DLOG((DERFC(A)- 
(B) )/ (DERFC(A- ALD) -DERFC(B- a12)) ))/R2 
ne (SAT et -R*S2*X(P+1))/(SLi*S2*DSORT((1.0D CR*A2) 42, ODO) ) 
B=(S2*T(P)-R ee S588 TADSORT (1. a -R**2)%*2.0D0)) 
as (A1*A1/2.0D0+DLOG(C)+DLOG((DERFC(A 
1 DERFC ENE A21)-DERFC(B-A21)))) a 


16 CONT 
TT(N)= (A2*A2/2. OD0+DLOG(c +DLOG(DERFC( (X(N) -R*T(N) )/ 
LDSORT((1.0DO-R**2)*2.0 
TOpREC (X(N) - -A1-R (TiN) 7Ad))/ 
IDSORT((1.0D0-R**2)*2.0D0)) 


“Haas (2(0- Bo eG’. aad 05).OR.(DABS(X(N)-XX(N)) .GT. 


=-05 )GO T 
10 CONTINUE 
X(N+1)=10.0D0 
APD=0 .ODO 
APF=0.0D0 
DO oT Cae N 


sorte 
ae 
APD= TAO) SODO*DCADRE ( 


F 
IF (IRE1 .NE. 0) WRIT oH 
( 





oe RERR,ERROR,IRE1) 
ERR, RERR,ERROR, IRE1) 


a 


APF=APF+O . SODO*DCADRE ( 
IF (IRE1 .NE. 0) WRITE 
81 CONTINUE 
PDL (M, I,N)=APD 
PF1(M.I. iN) =APF 
R3(M,1,N)=1.ODO0+C1(M)*PF1(M,1I,N)-PD1(M,1I,N) 
CARARKAKAKKKRARKKAKARRAKRAKKK RR KKKRRERRRRRRKKK RRR KEAEKKRKE 
C OUTPUT SYSTEM PARANETERS 
DO 120 J= 
WRITE(S, 62) xX(J),T(J) 
120 CONTINUE 
C WRITE(9,61)C,T(1),T(2) ,APD,APF 
C WRITE(8,61)APD, APF 
C WRITE(6,61)C,X(1),X(2),APD,APF,PDC,PFC 
20 CONTINUE 
i CONTINUE 
C WRITE(9,90) 
WRITE(8,90) 

500 CONTINUE 
CRRKRKRKAKKRKKRKRKKRKKKKKRKERKKKRKRKKRRKKKKKRKARKKKRKKRKRAKRKKKRKRKRKKARK 
C OUTPUT AVERAGE COST 

DO 501 J=1,2 
DO 502 I=1,20 
WRITE (10, 61)C1(I), ( R3 5) 
CAAKKKRRKKAAARRE Heke KARR KX Lee at ae x 
C OUTPUT PROB. OF DETECTION AND PROB. OF FALSE ALARM 
WRITE(8,61)( PD1(I,J,N),PF1(I,J3,N) ,N=1,5) 
502 CONTINUE 


ies 
6\60)IR 
ces 
6'60)IRE 


8% 


WRITE(8,90) 


CONTINUE 


FORMAT 1X,12,9(1K,F6.4)) 
FORMAT (2X, 'CON B | XXXAAAAAAAARARARAARKAKARKARKAAAAKKARARE | ) 


FORMAT oe eet 


FORMAT (1A, 20Rxe rio ec 
SLOF 


END 
FUNCTION F1(X) 
REAL*S X,F1,Al,A2,R, Xie, eye 
COMMON X11,R 
A1l=4.0D0 
A2=2.0D0 
FIL=DEXP(=({ X=Al ) #27 2eepey, 
1LDSQRT (8. ODO*DATAN(1.0D0O) ) 
F12=DERFC( (X11-A2-(X-Al oh 
1(DSQRT (2. 0D0*(1.0D0-R*%2) ) 
FL=F11*F12 
- RETURN 


END 

FUNCTION F2(X) 

REAL*8 X,F2,R,X11,F11,F12 

COMMON X11,R 

F11=DEXP (-X**2/2.0D0)/ 
1DSORT(8.ODO*DATAN(1.0D0)) 

F12=DERFC((X11-X*R) / 
1DSORT(2.0D0*(1.0D0-R**2) )) 

F2=F11*F12 

RETURN 

END 
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APPENDIX D 


PROGRAM LISTING TO CALCULATE PARAMETERS OF THE TWO 
QUANTIZERS FOR THE CASE OF EXPONENTIAL DISTRIBUTIONS 


Cc KAKKKKEKKRAKKKKEKRKKKRERRERKE 
C THIS PROGRAM CALCULATES THE OPTIMUM N-LEVEL QUANTIZERS OF TWO SEN- 
C SOR OBSERVATIONS OF EXPONENTIAL DISTRIBUTIONS 
C TO MINIMIZE A GLOBAL SYSTEM RISK FOR’ FUSION 
C SEE CHAPTER IV 
C KRRKERKKKKKKKKKRKKEKKARKERER 
REAL*8 X(8), eae Pani), 01( 6), P,P eee 9, 2r65,8 ae : ror D2 
a PF2,PDC,PFC,C1(20) C)X2. oe "APD, APF, R3 ),A 
1,APC(20,2) ,AFC(20,2), R4 (20, 2), AD AF2, oe 
INTEGER K, N, It IER1, IER2, POs 
DATA C1/50. ODO, 40. ODO, 30. SoBe 208 ‘ODO, 7. ae 10; ee 9.0D0; 
be.0D0,7.0D0,6. ODO, Ss ODO, 4. ODO, Sy5(0) 910) D2. OOF 1...50D 
Pie. ORO, -30D0 80D0,.70D0 60D0/ 
Cc KRARKKKEKRKRREKKKKRKKKAKKK 
C INITIAL VALUES pe QUANTIZER PARAMETERS 
C FIRST PSR GGe 
aAX(1)=00 .00000D0 
XA(2)=0.89000D0 ° 
XX(3)=01.50000D0 
XX(4)=1.89000D0 
m5 )=02.50000D0 
XA(6 )=3.09000D0 
C SECOND UANTIZER 
eT =3 .090000D0 
TT(2)=2.50000D0 
TT(3 )=01.89000D0 
TT(4)=1.50000D0 
TT(5)=00.89000D0 
TT(6)=0.Q00000D0 
DO 500 N=3,6 
BO 11 [=l1,2 


BeeeereuiP ERS OF THE EXP. DISTRISUTIONS 
AL=DFLOAT(I)*0.50D0 
DO” 20 M=1,.20 


C= er 
DO 55 P=1,N 
ee =DLOG(2. ODO*C) *DFLOAT (N- P+1)/DFLOAT(N)/AL 
XX (N- P+1)= DLOG(2. ODO*C) *DFLOAT (N- P+1)/DFLOAT(N)/AL 
55 CONTINUE 
K=0 
5 K=K+1l 


DO 89 PP=1,N 
eee) 
X(PP)=XX (PP 
89 CONTINUE 
CRAKRKKKKKAKAKRKAKRKRKKKKE 
C INPUT MAXIMUM NUMBER OF ITERATIONS 
MX=200 
Me (kK .GT.MX) G 10 
c XX(1)= DLO (2. ép0*C)/AL= mT) 
OK ! =0.0D0 
TIC )= (DLOG (DEXP (~AL*XK (1) )+DEXP (~ AL*XX(2)))+DLOG(2.0DO0*C) )/AL 


HK(B)=(DL0G (DEKE ee ee ALATT (Ps 1)}}sDL0G(2. La Uh 


ir (P)= DLOG( DEXP ( -AL*HX +DEXP (-AL*XX(P+1)))+DLOG(2.0D0*C) )/AL 
XX(N)=(DLOG(2. ODO*C) +DLOG ( DEXP (TT (N) )+DEXP (- AL*TT(N-1))))/AL 
C peer .ODO*C) /AL-XX(N) 


C ACCURACY CHECKING 


EE 


C INPUE PRECESSION HERE 
P=0 .10d-05 
ee -GT. AP).OR. (DABS(TT(1)=1(1) cue 


\co Toms 
10 CONTINUE 
WRITE(8,60) K 
BL=2.0DO*AL 
APD=0 .0DO 
APF=0.0D0 


0 81 Q=1,N-1 
APD&AED+DERE (~ALATT (9) ): (DERE (- BL ee 
APF=APF+DEXP(-BL*TT (OQ) )*(DEXP (-BL*XX (0) )-DEXP (-BL*XX(0+1 
81 CONTINUE 
APD=APO+DEXE (~ALSE N 


APF=APF+DEXP(-BL*K(N 
APC K, £)2(1 Ob0enRe (2 00S ae 
AFC(M,1I)=(1.0DO+DLOG( (4. 0D0*C) **2) )/(4.0D0*C) **2 
BOI (H, 1 .N)SABD 
PF1(M,I,N)=APF 
R3(M, I N)=1.0D0+C1(M)*PF1(M,I,N)-PD1(M,1I,N) 
Ra (M, TS=1.0D0+C1(M) *AFC(M, 1$-APC(M, I) 


DO en 
CRAKKARKKARKAKRAAKAKAKRAAKAKAKERKR 


C OUTPUT QUANTIZER PARAMETERS 
WRETE CS 62) 2 cs es) 
120 CONTINUE 
WReve (os 30) 


20 CONTINUE 
Pi CONTINUE 
VIRETE( 9, 90) 
c WRigE (990!) 


WRITE (8,90) 
500 CONTINUE 
DO 501 J=l72 
Bes02u 1-120 a 
AP2=1.0D0/(2.0D0*C1(I)) 
AF2=AP2**2 
RR2=1.0D0+C1 (I) *AF2-AP2 
CAKRAKAAKKKAKKKAAAAKRKAKARKARARKAARKRAKRE 
C OULPUL DETECTION Cosi 
“aq WRITE (10,61)C1(1) ,RR2,( R3(I,J,N) ,N=3,6),R4(T,J) 
CAKKKRKKKKKKAKAKAKAKKAKKRAKRAKARKRKKR 
C OUTPUT PROB. OF DETECTION AND PROB. OF FALSE ALARM 
WRITE(8,61)AP2,AF2,( PD1(I,J,N),PF1(I,J,N) ,N=3,5),APC(I,J) 
1 APCCIL > 
502 CONTINUE 
WRITE(8,90) 
501 CONTINUE 


60 FORMAT (1X,16,9(1X,F6.4)) 
90 FORMAT 2x, 'CON B KRARARKAARKKRKKKRKKRRKRKRKARKRARKRRKRRKRRKRRKRERKK | ) 
61 FORMAT 1 ee ene 
62 FORMAT (1X,3(1X,F15.8 
SOF 
END 
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APPENDIX E 


PROGRAM LISTING TO CALCULATE PARAMETERS OF THE TWO 


AANANANAN 


QUANTIZERS FOR EXAMPLE 3, CHAPTER IV 
} 


THIS PROGRAM CALCULATES THE OPTIMUM QUANTIZER PARAMETERS OF TWO 
N-LEVEL QUANTIZERS IN ORDER TO MINIMIZE A GLOBAL SYSTEM RISK FOR 
DETECTION OF SIGNALS WITH DIFFERENT VARIANCE. N=3 , N=5. 

Sl = SIGNAL VARIANCE UNDER Hl 
50 SIGNAL VARIANCE UNDER HO 


Tl = QUANTIZATION POINT FOR N=3 { A. =e ) 

X1,X2 QUANTIZATION POINTS FOR N=5 (X1,X2,-X1,-X2) 
C| (20) ARRAY OF RATIO OF COSTS 

K = Number of iterations. 


REAL*8 T1,T11,51,S2,R,C,X2,PD,PF,C1(15) ,231,221 ,X3,X33 ,X22 
1,TS,SSS,TIT(10) ,X1,PD3,PF3,PDC,PFC,R2,R3,RC 

INTEGER K,N,Kl 

DATA C1/01.0D0, .900D0, .800D0, .700D0,0.60D0,0.50D0,0.40D0 
10.30D0,0.20D0,0.10D0, .090D0, .080D0, .070D0 , 0.0600 ,0.050D0/ 

S1=1.0D 

SO=DSORT(2.0D0) 


INITIAL VALUES OF THE QUANTIZER PARAMETERS 


T1=01.15800D0 
X1=01.15800D0 
X2=00.15800D0 
WRITE(6,60)K,S1,S0,C 
WRITE(9,60)K,S1,S0,C 
WRITE(8,60)K,S1,S0,C 
SSS=(1.0D0/SO**2-1.0D0/S1**2)/2.0D0 
DO 101 N=2,2 
DO 100 I=1,15 
cei 0) 
K=0 
K=K+1 
Mo henGr 100) Ge To: 10 
Tibi T 1 
TS=(DLOG(S1/S0)+ 
Bros (5 Jo DL0G (CDERF (T1/DSQBT | 
1/SO))/(DERF(T1/DSORT(2.0D0) 
[mets .GT. 0:0D0) TI=DSORT(T 
T1=DSORT (DSORT(TS) ) 
IF((DABS(T1-T11) .GT. 0.10D-05)) GO To 5 








2.0D0) 
ay DFLOAT(N-1))/SSS 


CONTINUE 
10 TIT(N)=T1l 
K1=0 
55 K1=K1+1 

i (Xl. GT:100) GO TO 15 
K33=X3 
X22=K2 
X2=(DLOG(S1/S0)+ 

Eee ace coon cis ba 

50) )/(DERE(X3/DSORT(2.0D0)/S1))))/SSS 

Cc IF (TS .GT. 0.0D0) T1l=DSQRT(TS) 
X2=DSORT (X2) 

Z30=DERF (X3/DSQRT(2.0D0)/SO 
Z20=DERF(X2/DSORT(2.0D0)/SO0 
Z31=DERF(X3/DSORT(2.0D0)/S1 
Z21=DERF(X2/DSORT(2.0D0)/S1 
X3=(DLOG(S1/S0)+ 

IDLOG(C)+DLOG( (Z30-Z20)/ (Z31-Z21)))/SSS 
X3=DSQRT (DABS (K3 
IF(((DABS(X2-K22) .GT. 0.10D-05)) .OR.(( DABS(X3-X33) .GT. 

10.10D-05))) GO TO 55 

CONTINUE 
15 mor (N)=T 1 I 


ie 


TS=-DLOG(S0**2/S1**2/C)/SSS 
PDC=1.0D0- ~DEXP (- ie On Sa 
PFC=1.0D0-DEXP(-TS/(2.0D0*S0*SO 
RC=1.0D0+C*PFC-PDC 
WRITE(9,60)K,T1,T11,X2,X22,X3,X33 
WRITE (6,60)K,T1,T11 
PD= =SDERE (711 /DSgaaiae oooh 
PF=DERF (T11/DSORT(2.0D0)/SO)**N 
R2=1.0D0+C%PF-PD 
PD3=(DERF (X22/DS RT 2.0D0)/S1)*2.0D0-DERF (X33/DSQRT(2.0D0)/S1)) 
1 *DERF (X33/DSORT(2.0D0)/S1 
PF3=(DERF (X22/DS RT 4. 0D0)/S0)*2. ODO-DERF (X33/DSQRT(2.0D0)/S0)) 
1 *DERF (33/0808 ra SO) 
+C OF Pb 
REL Teye -60)N,PDC,PFC,PD,PF,PD3,PF3 


WRITE(10, 60)N, /C,R2,R3,RC 
C iF ((r bo. meen. (i -EQ. 10 )) WRITE(10,60)N,C,R2,R3,RC 
WRITE(6,60)N,C,TIT(N) ,PD,PF,PD3,PF3 
100 CONTINUE 
101 CONTINUE 
60 FORMAT (1X,I3,6(1X,F10.7)) 


END 
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APPENDIX F 
LINEAR MINIMUM MEAN SQUARE ESTIMATE OF Y, 


Having Y =[ Vig Y2 |' the LMMS estimate of y, and the corresponding mean 


Square error are given by [34]: 


A 


ere E(y, X°)E{Z y'}' ¥ (F.1) 
and 
E{e* }= Efy,?)-E{y, Y*}E(¥ x}! E{y y, } (F.2) 
where 
E{y, 2 }=(Ety, y,,) Ely y2}1 (F.3) 
and 
iy Y'} oe oy ) | (F.4) 


The entries of these matrices are: 


i, ay Q EY, Ng =Q) (F.5) 
a 
iF a Q S 
er aN 
E{Y2 Vig I= 2 Q Ely. Nig =Q} (F.6) 


but 
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EtY2 (Vig = Q JH EUYo/ ¥, Wg = Qh 


For the case where y, and y, are jointly gaussian, we can write 
= 2 
E{y, Vig }=po, oy P Q; C/s, 


Ety, Yig }=p 6, 64H 


> - { 
EtYig } =>, Q, 


nae 
= ane 


(F.7) 


(F.8) 


(F.9) 


(F.10) 


Inserting these in (F.1 ) and (F.2 }) and performing matrix multiplications yields 


A 


| 0 
os g Po yig  P cee leDs v3] ((n -W? p*) 
Zz 


and 
Efe? } =o," (1-p? )(1-@ )/(1-p? @ ) 
where @ is given by: 


Dh 
oO =" /N. 


22 


(6.2) 


(6.11) 


(6.12) 


APPENDIX G 


SOLUTION OF THE LLOYD-MAX QUANTIZER PARAMETERS BY 
THE METHOD OF SUCCESSIVE SUBSTITUTION 


lL INTRODUCTION 

The minimum distortion quantizer parameters [18,23], as well as parameters 
based on other criterion such as quantizers for signal detection [41], minimum risk 
“quantizers and quantizers for LMMS estimation error dealt with in this thesis, can be 
solved by Max’s trial and error technique [18]. There are also many other 
approximation methods to calculate the quantizer parameters [42], [43] and [44]. 

In this Appendix we apply the method of successive substitution and its 
modifications [19] to solve for the Lloyd-Max quantizer parameters. It is more accurate 
and computationally more efficient than the previously reported methods. It is shown 


to easily generate 7 bit (128 level) optimum quantization. 


2, STATEMENT OF THE PROBLEM 
The Lloyd-Max minimum mean square distortion quantizer problem deals with 
transforming a random variable X of differentiable probability density function f(x) 


into the N-level discrete random variable Y. 
nex) = Y. for X & [x, ey ] (G.1) 


The optimum parameters minimize the distortion D 


N Lx. 
Bee St bx-y, 2 7x) ax Ge 
2a) (xy, )° fox) (G.2) 
with 
-O =x, Sx, 5..5 Xv Ss A ro 
Differentiating D with respect to-x, and y, yields the following necessary conditions of 
optimality : 
ee oY.., )/2 1=2,3,...N (G3) 


Zs 


y, =(f ae ET x)dx)/(f _ Ax)dx ) i= 1,2,..N (G.4) 


I 


a set of simultaneous equations of propagating character. That is, if y, is chosen 
correctly then x, can be calculated from (G.4 ), y, from (G.3 ), x, from (G.4 ) and so 
forth [18]. In this case the value of y,, calculated from (G.3 ) must agree with its value 
calculated from (G.4 ) with x ,, =. This was the core of Max's trial and error 
algorithm; to pick a value for y, and calculate the parameters up to and including y,, , 
which must agree with the value of y,; calculated from (G.4 ), otherwise, to pick 


another value of y,. Let us put the system of equations in the form 


Z =€ (Z) (G.5) 


where Z is a 2N-1 vector given by: 
=e t 
Z =I oe X5 Va cee YN (G.6) 
and apply the iterative substitution 


ce ea) 7 


“new 


with a suitable initial guess. The convergence is guaranteed if d G,. /a Z. is sufficiently 
Smidllefor every Kij— 12:2, 2-1 | 1p. Prem ce ae 


o G, /d yy = UX 44 “¥} MICS 4 )+(y; “ MIC, \i(2P, ) (G.8) 


where P. is the probability the input of the quantizer is in the j,, interval. 


Ne 
Pe cian: (G.9) 
aT ee 
J 


The numerator in (G.8 ) is an approximation of the integral in (G.9 ) by the 
trapezoidal rule with the subdivision x, Yi Xa) ], so the value of the derivative is 
very likely less than one. Also, substituting for y; and Yin in (G.3 ) from (G.4 ) and 


differentiating with respect to x, It is easily to show that 
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8G, /8 x, =(y, -x, Rx, /(2P.) +(x, -¥,. Mx, )/2P (G.10) 


which is less than (0 G; /o y;). The method can be more efficient if we use the updated 
values in the same iteration. In this modification of the method the best current 
values of the parameters are used. This choice may also enhance convergence. The 
method also avoids the tedious calculation of the upper limit of the integral to solve for 


fine ext x; in (G.4 ). 


3. NUMERICAL RESULTS 

We have solved for the quantizer parameters for a gaussian random variable of 
zero mean and unit variance for several values of N up to 128. Also the mean square 
error D and the output entropy eur P, log, (P, )) have been calculated. The results 
presented in Table 4 show that in several cases Max’s results, which were only 
available up to N = 36, are not accurate in the last digit. 

Key to Table 4 
The numbering in the table is as follows. 


1. For N even, each table begins with the (N/2+1),, parameters. In this case the 
(N/2+ 1), value of x is Zero. 


2. For N odd, Each table begins with the (N/2+2),, parameters. In this case the 
(N/2+ 2), value of y is zero. 


Negative parameters can be calculated from the symmetry relation 


A> Xa jt 


(G.11) 
and 


‘i = ae (G.12) 


A FORTRAN program to calculate the parameters ,distortion and entropy 


follows Table 4. The only input to the program is N, the number of quantization levels. 


125 


TABLE 4 
MAX’S QUANTIZER PARAMETERS FOR THE NORMAL DISTRIBUTION 
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TABLE 4 
MAX’S QUANTIZER PARAMETERS FOR THE NORMAL DISTRIBUTION (CONT 'D.) 
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4. PROGRAM LISTING TO CALCULATE THE LLOYD-MAX QUANTIZER 
PARAMETERS 


C THIS PROGRAM CALCULATES LLOYD-MAX QUANTIZER PARAMETERS BY THE METHOD 
C OF SUCCESSIVE SUBSTITUTION FOR THE NORMAL DISTRIBUTION OF ZERO MEAN 
C AND UNIT VARIANCE 
C The INPUT TO THE PROGRAM IS 
i THE NUMBER OF QUANTIZATION LEVELS N 
2 THE MAXIMUM NUMBER OF ITERATIONS M 
2 THE ACCURACY 
KKRKKRKKKRKARKKRKRKARKKRKRAARKKRKRKKKERE 
REAL*8 X(199),T(199) ,XX(199) ,TT(199),C ,DELTA,AP(199) ,AP 
1, ERROR , ENTROP 
INTEGER K,N,1I,P,N1,N2,N3,M 
C=DSQRT(00.50D0/DATAN(1.0DO)) 
DO 99 N=110,110 
CARA KARKKKAAKAKRARARRARARKAARARARAREARRARRARARKRARAARARRARARARAKK 


c ean han NUMBER OF QUANTIZATION LEVELS 


Whiten Joo 
WEELE (> 66 
WRITE(9,67 
Wine Cibo 
CRAAKAKAKAKRARARKRAKRK AAR RAR AKA RA RRA RARARRRKKAKRARARR RARE RARKKAR 


: INITIALIZATION OF THE QUANTIZER PARAMETERS 


DELTA=0.0150D0*DFLOAT(N) 
ee 


AAAN 


TT(1)=-5.50000D0 

ue See 

T(1)=TT(1 

po 50 L=2.N 
Tih otra oe 
¥x(L)=(TT(1)+TT (1-1) )/2.0D0 
Sth 
T(LJ=TT(L 


50 CONTINUE 
CAKKKKKKAKAKKAKRAKKRKRKKKAKRRKKRRKAKRKRKKKRKKKRKRRKRKKRARKRKERRR 
E BEGINING OF THE ITERATIONS 
C M = MAXIMUM NUMBER OF ITERATIONS 

M = 1050 
K=0 
5 K=K+1 
TE (GT i Goeron lo 
IF (K .GT. 1)X(N)=KXX(N 
if (Kk {Cla eer 
TT (1)=-C*DEXP ( -XX(2) *XX 2) (2-000 )7 (RIES ages 
IDERFC(XX (2) /DSQRT 2.0D0)) 
T(1)=TT ( 


TE°(, NeEOs 2 iGo, Tora 
DO 15 P=2 : 1 


XX(P)=(T(P)+T(P-1))/2.0D0 
X(P)=XXK(P) 
ae Cae en 7s IRSGbnTe GOUeGEnE en te 
; soa =TT(P)*C/ (DERFC(X(P)/DSQRT(2.0D0) )-DERFC(X(P+1)/DSQRT(2.0 
D 
eee 
LS CONTINUE 
7 CONTINUE 
SD CAL GIE CEN ROB) 
TN aoe ee ONY er) 2.0D0)*C/DERFC(XX(N)/DSQORT(2.0D0) ) 
T(N)=TT(N 
Nee SEA eaioR Te ye 
N1=IDINT(DFLOAT( (N+1)/2 
CAKRKKKKARAKRARKRKRAKKRAKRKAKKKRK KRAERKRKKKKKKKKKKKKKKRRKKKKEK 


c CHECKING THE PRECISION OF SHE SOLUTE 
e AP = REQUIRED ACCURACY 
AP=0.10D-6 
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FF cone 2 eos). AND. eae 


(iz eGr. oh?) )S9 TO 5 
N+1,2) -EQ. 0).AND. (DABS(T(N1)).G 
NUE 


): 
T(N1 


IF((MOD . AP))GO TO 5 
ONTI 
10 CONTINUE 
CARKAKKKKKKKKKKKKK KAEKKKKREKKRKEKKRKKKERKKEKAARREKRAKRKEREKRERE 
€ OUTPUT RESULTS 


IF HOD (N 2) .EQ. 0)N3=N2 
IF (MOD(N+1,2) .EQ. 0)N3=N1 
WRITE(6,60) K 
DO 120 J=1, NS 
peel .EO. 1) 
1 WRITE(9,71)J, Ted) 
1 WRITE(9,61)J, X(J),T(J) 
120 CONTINUE 
s ea =10.0D0 
X(1)=-10.0D0 
ERROR=0 .ODO 
ENTROP=0.0DO 
DO 222 I=1,N 
az (T)= RPE) 2 00ers .ODO) )-DERFC(X(I+1) /DSQRT(2.0D0) ) 
AP(I)=AP(1I)/2.0D0 
ERROR=ERROR+AP(I)*T(1)* 
ENTROP=ENTROP-AP (I) epee. ODO) 
222 CONTINUE 
ERROR=1.0D0-ERROR 


WRITE(9,66 
WRITE(9,62) ERROR 
WRITE(9,66 
WRITE 9/63 ENTROP 
WRITE(9,66 
WRITE(6,72) K 
WRITE(9,9 
WRITE(9,66 
c99 CONTINUE 
65 FORMAT(3X,' N= ',17) 
66 FORMAT ( 3X «| aw a0 ae ae ne ow 0 ow ow ow ow ow an aw ow ono on on ow on ow on ao oo 
67 FORMAT 3X, 7 X y 
60 FORMAT(1X, ‘17 ,Oo(1X,F6.4)) 
90 FORMAT (2X, \ paosaesnecssaesesseesasssassssassas=ss====2! ) 


ou FORMAT(1K,1I4, 2(2X,F9.6)) 
ie) FORMAT(14%,14,11%, 2 Hono) 


62 FORMAT ee 'ERROR =! 2(1K, Boor) 
72 FORMAT (3X, '# ITERATIONS = ! 
oe FORMAT ai" PSVgRorye="> Ae Bore) 
Soe 
END 
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